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1.  INTRODUCTION  AND  SUMMARY 


The  information  in  this  section  provides  a  brief  background  on  the  IMPACT  Program,  the 
objectives  of  the  advanced  technology  experiment,  the  scope  of  this  report,  and  a  summary  of  the 
procedures  and  results  of  the  experiment. 

1.1  Background 

The  Wright  Laboratory  Advanced  Cockpits  Branch  (WL/FIGP)  Integrated 
Mission/Precision  Attack  Cockpit  Technology  (IMPACT)  program  focuses  on  determining 
cockpit  and  pilot-vehicle  interface  (PVI)  requirements  for  a  single-seat,  multi-role  fighter  aircraft 
performing  at  night  and  in  adverse  weather.  During  the  Mission  Analysis  and  Interface 
Requirements  phase  of  the  IMPACT  Program,  Air  Combat  Command  mission  requirements  and 
current  technologies  were  assessed.  Based  on  the  requirements  and  the  associated  high  workload 
segments,  the  air  interdiction  (AI)  scenario  was  selected  by  the  IMPACT  team  as  the  starting 
point  for  analyzing  the  overall  IMPACT  cockpit  design  requirements.  The  F-15E  two-seat  strike 
fighter  was  identified  as  the  baseline  weapon  system  because  it  is  currently  the  US  Air  Force’s 
most  capable  aircraft  in  performing  a  multi-role  mission  in  adverse  weather  and  at  night. 

Through  mission  decomposition  and  analysis  the  IMPACT  team  determined  that  the 
ingress,  attack,  and  egress  segments  of  the  AI  mission  contained  the  highest  levels  of  workload 
and  therefore  warranted  the  greatest  attention.  With  the  mission  segments  identified,  both 
function  and  task  analyses  were  conducted  to  understand  the  allocation  of  required  functions  for 
the  pilot,  the  weapon  systems  officer  (WSO),  and  the  aircraft  systems  for  the  F-15E  platform. 
Mission  function  timelines  were  developed  to  graphically  determine  which  areas  of  the  mission 
required  multiple  functions/tasks  to  be  accomplished  simultaneously.  In  addition  to  the  two-seat 
F-15E  mission  timeline,  which  includes  a  pilot  and  a  WSO,  a  separate  mission  timeline  was 
developed  for  a  conceptual,  single-seat  version  of  that  system.  To  facilitate  the  transition  to  a 
single-seat  aircraft,  it  was  essential  for  the  experiment  team  to  understand  how  these  functions 
were  allocated  in  the  two-seat  system. 
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By  understanding  current  requirements  and  limitations,  the  IMPACT  team  performed  an  in- 
depth  review  of  advanced  technologies  to  identify  possible  solutions.  These  technologies 
included  advances  in  sensors,  controls  and  displays,  weapons,  and  pilot  aiding  techniques.  The 
evaluation  of  these  technologies  led  to  the  Preliminary  Cockpit  Design  and  Trade  Studies  event. 
The  design  objective  of  this  event  is  to  develop  a  cockpit  that  exhibits  equal  or  better  operational 
and  performance  capabilities  when  compared  to  the  F-15E  in  the  AI  mission.  Advancements  in 
cockpit  technology  will  be  integrated  into  the  IMPACT  cockpit  to  achieve  improvements  in 
operational  capability. 

The  first  IMPACT  experiment  was  a  Role  Playing  Exercise,  whose  objectives  were  to  (1) 
identify  workload  associated  with  an  AI  mission  at  night  under  adverse  weather,  (2)  identify 
possible  cockpit-related  technologies  that  could  improve  the  overall  effectiveness  of  a  single-seat 
fighter  performing  an  interdiction  mission  at  night  under  adverse  weather,  and  (3)  verify  the 
“Role  Playing”  methodology.  Workload  levels  associated  with  an  AI  mission  at  night  and  under 
the  weather  were  determined,  and  advanced  technologies  were  identified  that  had  the  highest 
potential  for  reducing  workload,  increasing  situation  awareness  (SA),  and  improving  mission 
effectiveness.  Four  technologies  that  were  determined  as  having  the  greatest  potential  were 
sensor  fusion,  data  link,  head  steered  sensor,  and  the  helmet-mounted  display  (HMD) 
(Montecalvo  et  al.,  1994). 

1.2  Objectives 

This  Advanced  Technology  Integration  Experiment  focused  on  three  technologies:  an 
HMD,  large  tactical  situation  display  (TSD),  and  directional  audio.  Each  technology  was 
integrated  into  the  IMPACT  cockpit  and  its  ability  to  aid  a  pilot  in  visually  acquiring  surface  and 
air  threats  and  ground  targets  was  assessed. 

The  overall  objectives  of  this  experiment  were  to 

•  Evaluate  pilot  threat  acquisition  performance  in  two  cockpit  configurations:  a  baseline 
cockpit  (based  on  the  F-15E,  and  hereafter  referred-to  simply  as  the  Baseline)  containing 
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the  formats  necessary  to  accomplish  the  experimental  tasks,  and  the  IMPACT  cockpit 
containing  the  HMD,  directional  audio,  and  10”  x  10”  TSD  advanced  technologies. 

•  Evaluate  pilot  flight  performance  while  attacking  ground  targets  in  each  cockpit 
configuration. 

•  Evaluate  pilot  workload  and  SA  for  both  threat  acquisition  and  weapon  delivery  for  each 
cockpit  configuration. 

•  Identify  strengths  and  weaknesses  of  the  IMPACT  configuration  and  the  individual 
advanced  technologies,  using  post-mission  and  post-experiment  questionnaires. 

Expectations  were  that  the  advanced  technologies  incorporated  into  the  IMPACT  cockpit 
would  result  in  improved  performance  for  acquisition  of  surface  and  air  threats.  One  hypothesis 
was  that,  because  of  multiple  sources  of  threat  cueing,  the  time  to  acquire  threats  would  be  lower 
for  the  IMPACT  cockpit  as  compared  to  the  baseline.  Also,  expectations  were  that  the  IMPACT 
cockpit  would  result  in  smaller  flight  parameter  deviations  as  compared  to  the  baseline  because 
the  multiple  sources  of  cueing  would  facilitate  quicker  threat  acquisition  outside  the  HUD  field- 
of-view  (FOV),  thus  allowing  the  pilot  to  quickly  return  to  the  flying  task. 

Another  hypothesis  was  that  the  IMPACT  cockpit  would  result  in  smaller  groundspeed, 
flight  path,  and  attack  axis  deviations  than  the  baseline  during  the  Weapon  Delivery  Phase. 
Through  the  use  of  the  HMD  in  the  IMPACT  cockpit,  the  pilot  would  have  a  greatly  expanded 
FOV  as  compared  to  the  simulated  baseline  Low  Altitude  Navigation  and  Targeting  Infra-Red  for 
Night  (LANTIRN)  HUD.  This  expanded  FOV  was  expected  to  facilitate  optimized  aircraft 
maneuvering  for  a  faster  and  smoother  alignment  onto  attack  parameters. 

Finally,  it  was  expected  that  workload  would  be  lower  for  IMPACT  than  the  baseline  during 
threat  acquisition  and  weapon  delivery.  The  anticipated  reduction  of  workload  when  acquiring 
threats  and  attacking  targets  can  be  attributed  to  the  advanced  technologies. 
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1.3  Scope  of  Report 


This  report  provides  descriptions  of  the  advanced  technologies  evaluated  in  this  experiment, 
an  explanation  of  the  methodology  employed  to  analyze  those  technologies,  the  results  of  the 
experiment,  and  a  discussion  of  those  results. 

1.4  Executive  Summary 

Three  advanced  technologies  were  integrated  into  the  IMPACT  cockpit:  an  HMD, 
directional  audio,  and  a  10”  x  10”  TSD  incorporating  Real  Time  Information  In  The  Cockpit 
(RTIC).  The  ability  of  these  technologies  to  aid  a  pilot  in  visually  acquiring  surface  and  air 
threats  and  ground  targets  was  assessed  because  in  two-seat  fighters,  the  pilot  and  the  WSO  share 
responsibility  for  these  tasks. 

The  subject  pilots  were  briefed  on  the  experiment  and  were  trained  in  the  simulator  to  fly 
both  cockpit  configurations  (baseline  and  IMPACT).  At  the  completion  of  training,  they 
participated  in  eight  data  collection  runs.  The  scenario  was  a  night,  under  the  weather 
interdiction  mission  to  attack  a  SCUD  missile  launcher.  The  pilots  flew  a  pre-briefed  route  to  a 
pop-up  weapon  delivery.  Along  the  route,  they  were  tasked  to  visually  acquire  threats  that  had 
locked  onto  the  aircraft.  The  pilots  reported  their  workload  during  the  mission  using  the 
Subjective  Workload  Assessment  Technique  (SWAT).  In  addition,  flight  performance  data  were 
collected  during  each  simulator  run. 

Data  analysis  consisted  of  the  determination  of  threat  acquisition  time,  threat  acquisition 
success  rate,  flight  performance  during  threat  acquisition  and  weapon  delivery,  workload,  and 
SA.  Questionnaires  and  interviews  focused  on  the  comparison  of  the  two  cockpits  and  the 
influence  of  each  individual  technology  on  the  overall  IMPACT  objectives. 

The  results  of  this  experiment  show  that  the  implementation  of  the  HMD,  directional  audio, 
and  large  TSDs  decreased  the  amount  of  time  required  for  the  pilot  to  visually  acquire  threats. 
Also,  approximately  20%  more  threats  were  acquired  and  pilot  workload  was  lower  in  the 
IMPACT  cockpit.  The  pilots  also  reported  that  their  situation  awareness  was  higher  in  the 
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advanced  cockpit  than  in  the  baseline  cockpit.  In  the  area  of  flight  performance,  pilots 
encountered  fewer  deviations  in  altitude,  airspeed,  and  heading  in  the  IMPACT  configuration. 
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2.  DESCRIPTIONS  OF  ADVANCED  TECHNOLOGIES 


For  a  pilot  operating  a  single-seat,  multi-role  fighter  to  be  as  effective  as  an  F-15E  crew 
in  the  night  interdiction  mission,  advanced  technologies  are  necessary  to  perform  the  inflight 
tasks  currently  accomplished  by  the  F-15E  WSO.  The  technologies  must  aid  the  pilot  in  threat 
reaction,  target  designation,  and  weapon  delivery.  The  integration  of  the  technologies  should 
decrease  high  workload  demands  on  the  pilot  while  increasing  SA. 

An  HMD,  3-D  audio,  and  a  large  TSD  were  identified  in  a  previous  experiment 
(Montecalvo  et  ah,  1994)  as  having  the  potential  to  assist  a  pilot  in  a  single-seat,  multi-role 
aircraft  attempting  to  acquire  surface/air  threats  and  ground  targets.  The  appropriate  application 
of  these  advanced  technologies  may  provide  the  single-seat  pilot  with  equal  or  better  SA  with 
respect  to  threat  location  than  is  now  possible  with  two  crew  members. 

2.1  Helmet-Mounted  Display 

An  HMD  can  be  beneficial  in  replacing  the  visual  search  capability  that  is  lost  when  crew 
size  is  reduced  from  two  to  one.  In  many  cases,  a  quick  visual  acquisition  of  an  attacking  fighter 
or  a  surface-to-air  missile  in  flight  is  absolutely  essential  to  survival. 

Literature  reviews  were  conducted  to  gather  information  concerning  the  current  state  of 
development  and  recommended  mechanization  for  each  advanced  technology.  A  significant 
amount  of  work  has  been  performed  in  the  area  of  HMDs,  making  it  necessary  to  limit  the 
literature  review  to  programs  and  studies  that  contain  (or  define)  functional  and  operational 
design  criteria.  In  addition,  through  in-house  meetings,  the  decision  was  made  to  focus  only  on 
those  items  that  were  necessary  to  help  facilitate  off-boresight  cueing,  tracking,  and  designation. 

Advanced  technologies  have  the  potential  to  assist  the  single-seat  pilot  during  night 
weapons  deliveries.  When  Low  Altitude  Navigation  and  Targeting  Infra-Red  for  Night 
(LANTIRN)-equipped  fighters  execute  level,  climbing,  and  diving  visual  weapon  deliveries  at 
night,  using  the  forward  looking  infra-red  (FLIR)  for  target  acquisition,  they  must  fly  directly 
towards  the  target  for  the  duration  of  the  weapon  delivery  maneuver  because  of  the  FLIR’s 
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narrow  field-of-view  (FOV).  Therefore,  the  pilot  flies  a  very  predictable  path  to  the  target, 
increasing  the  probability  that  enemy  threat  systems  can  engage  the  aircraft.  One  alternative 
currently  used  in  daylight  visual  flight  rules  (VFR)  conditions  is  to  perform  an  off-axis  attack 
where  the  pilot’s  weapon  delivery  axis  is  not  necessarily  aligned  with  the  approach  course.  This 
alternative  complicates  the  enemy’s  problem,  but  is  very  difficult  to  accomplish  at  night  with 
today’s  technology.  The  incorporation  of  advanced  concepts  such  as  HMD  with  a  head-steered 
FLIR  may  provide  the  pilot  sufficient  SA  to  allow  off-axis  target  attack  without  increasing 
workload. 

HMD  symbology  was  developed  based  on  the  following  operational  requirements:  early 
visual  acquisition  of  threats/targets;  maintaining  visual  acquisition  of  threats/targets;  maintaining 
visual  acquisition  of  friendly  aircraft;  and  the  capability  to  closely  monitor  ownship  energy  state. 
These  operational  requirements  were  consistent  with  those  determined  by  operational  fighter 
pilots  (Barnes,  1 989).  Based  on  these  requirements,  the  working  group  identified  symbology  and 
mechanizations  using  the  Night  Attack  Program  Pilot-Vehicle  Interface  Studies  (Royer,  1991  and 
1992)  as  initial  references.  Once  the  symbology  and  the  overall  mechanizations  were  selected, 
Simulation  Project  Modification  Requests  (SPMRs)  were  developed  to  document  the 
functionality  and  design  specifications  for  both  cockpits  and  their  display  formats.  Appendix  A 
provides  a  copy  of  the  SPMRs  used  for  this  experiment. 

2.2  3-Dimensional  Audio 

Potential  benefits  of  3  dimensional  audio  are  detection  and  identification  of  threats, 
targets  and  wingmen,  as  well  as  navigation  aids  such  as  terrain  alerts.  The  current  drive  towards 
displaying  real-time  information  requires  additional  complex  visual  displays  that  further  increase 
the  perceptual  and  cognitive  demands  on  the  pilot  (Dolan  et  al.,  1993).  To  address  this  issue, 
other  modalities  such  as  the  auditory  channel  are  being  investigated  in  an  attempt  to  decrease  the 
demand  on  visual  resources. 

The  benefits  of  3  dimensional  audio  centers  around  two  issues.  First,  it  may  provide  a 
more  intuitive  indication  of  threat  location  with  respect  to  the  pilot’s  head.  Second,  proper 
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integration  should  reduce  demand  on  the  pilot’s  visual  resources.  This  reduction  should  be 
realized  because  concurrent  tasks  that  share  the  same  pool  of  resources  (e.g.,  two  visual  tasks) 
tend  to  interfere  with  each  other  more  than  concurrent  tasks  that  do  not  share  the  same  resources 
(e.g.,  a  visual  task  and  an  auditory  task)  (Wickens,  1984). 

Directional  audio  systems  encode  naturally  occurring  spatial  information  into  an  audio 
signal  and  present  a  sound  source  that  can  vary  in  azimuth  over  stereo  headphones  incorporated 
into  the  pilot’s  helmet.  To  the  pilot,  the  encoded  signal  sounds  as  if  it  is  originating  from  a 
particular  location.  As  the  pilot  changes  head  orientation,  the  relative  position  of  the  sound 
source  is  updated  to  make  the  sound  appear  stationary  with  the  listening  environment.  Patterson 
et  al.,  (1993)  surveyed  76  experienced  military  pilots  regarding  the  application  of  3-D  audio. 
Most  pilots  preferred  localized  audio  signals  to  the  present  non-localized  audio,  especially  for 
wingman  communications  and  threat  audio.  Based  on  Patterson’s  research,  a  directional  audio 
system  that  provided  localized  tones  for  threat  warnings  was  incorporated  into  this  IMPACT 
experiment. 

2.3  Large  Tactical  Situation  Displays 

Until  recently,  display  technology  has  been  limited  by  smaller  (5”  x  5”  or  4”  x  4”) 
displays.  In  an  effort  to  promote  greater  SA,  a  large  10”  x  10”  TSD  was  developed  for  this 
experiment.  The  large  TSD  simulates  the  advanced  techniques  of  fusing  data  from  multiple 
sensors  to  present  a  composite  set  of  data  on  a  single  display  format  without  introducing 
unacceptable  symbology  clutter  as  in  a  4  or  5  inch  display. 

TSD  symbology  and  color  coding  were  developed  and  incorporated  into  the  IMPACT 
cockpit.  SPMRs  were  developed  to  document  the  functionality  and  design  specifications  of  each 
proposed  format  (see  Appendix  A).  The  TSD  formats  were  consistent  with  display  elements 
reported  in  research  conducted  by  Way  et  al.,  (1987),  Galvin  and  Pecha  (1992),  and  Vanderbach 
et  al.,  (1993) 
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3.  METHODOLOGY 


In  this  experiment,  subject  pilots  flew  a  simulator  that  was  reconfigurable  as  both  the 
baseline  and  the  IMPACT  cockpits.  Advanced  technologies  were  integrated,  subjects  were 
trained,  and  data  were  collected  in  the  simulator  and  through  questionnaires  and  post-mission 
interviews.  Measures  of  effectiveness,  performance,  workload,  situation  awareness  (SA),  and 
subjective  data  were  determined  and  analyzed. 

3.1  Subjects 

Eighteen  male  pilots  participated  as  subjects.  Total  operational  aircraft  flight  time  for  the 
pilots  ranged  from  770  to  3470  hours  (mean  =  2618).  Sixteen  of  the  pilots  had  fighter  aircraft 
experience  and  two  had  B-52  experience.  Appendix  B  provides  more  details  concerning  pilot 
flying  experience. 

3.2  Apparatus 

The  apparatus  consisted  of  the  cockpit  simulator,  the  three  advanced  technologies  (HMD, 
TSD,  and  directional  audio),  the  head  tracking  device,  the  out-the-window  visual  scene 
projection  screen,  and  the  experimenter’s  station.  Materials  used  to  train  the  subject  pilots  and 
the  post-mission  questionnaires  were  also  part  of  the  experiment  apparatus. 

3.2.1  Cockpit  Simulator 

The  evaluation  was  conducted  in  the  Crew  System  Integration  Laboratory’s  (CSIL) 
Manned  Combat  Station  (MCS),  which  was  reconfigurable  as  either  the  baseline  (F-15E  front 
panel)  cockpit  or  the  IMPACT  cockpit  (see  Figure  1).  The  MCS  was  a  single  seat  cockpit 
“shell”  about  the  size  of  an  F-16.  A  single  Matsushita  27”  color  monitor  presented  the  head- 
down,  dynamic,  non-interactive  display  formats  for  both  cockpit  configurations.  Flight  control 
and  limited  hands-on-throttle-and-stick  (HOT AS)  switching  was  provided  via  F- 15  type  throttles 
and  a  force  side-stick  controller  with  an  F-15  grip  (see  Appendix  A,  SPMR  1.1). 
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Figure  2  shows  the  baseline  cockpit  configuration  (TO  1F-15E-1,  1993).  Each  frame 
consisted  of  the  F-15E  up-front  control  (UFC,  top  center  in  figure),  left  multi-function  display 
(MFD),  center  MFD,  and  right  MFD.  Additional  displays  included  the  fuel  quantity  and  engine 
instrument  gauges.  The  frames  were  developed  using  Designer  software,  and  were  displayed  on 
the  monitor  using  a  Silicon  Graphics  Onyx  Reality  Engine  2  system.  F-15E  heads-up  display 
(HUD)  symbology  (TO  1F-15E-34-1,  1993)  and  the  outside  world  forward-looking  infra-red 
(FLIR)  image,  also  generated  by  the  Silicon  Graphics  system,  were  presented  on  the  Barco 
projection  system.  Only  those  controls,  displays,  and  formats  necessary  to  perform  the  threat  and 
target  acquisition  tasks  for  the  experiment  were  available  for  pilot  use. 
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Figure  1.  CSIL  MCS  Flight  Simulator 
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Figure  2.  Baseline  Cockpit  Configuration 


Figure  3  shows  the  IMPACT  cockpit  configuration.  Two  10”  x  10”  displays  and  an 
upfront  controller  were  presented  on  the  head-down  monitor.  F-15E  PIUD  symbology  and  the 
outside  world  FLIR  image  were  presented  on  the  Barco  projection  system.  The  IMPACT 
cockpit  also  integrated  the  HMD  and  directional  audio  system.  The  formats  used  in  the 
experiment  and  the  associated  operational  descriptions  are  provided  in  Appendix  A. 
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Figure  3.  IMPACT  Cockpit  Configuration 
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3.2.2  Helmet-Mounted  Display 


The  HMD  was  a  monochrome  Kaiser  SIM  EYE  40  consisting  of  a  helmet,  a  binocular 
HMD,  and  an  electrical  interface  unit.  The  system  field-of-view  (FOV)  was  40°  circular  with 
100%  overlap.  The  cathode  ray  tubes  (CRTs)  were  driven  at  a  rate  of  60  Hertz,  non-interlaced 
with  a  resolution  of  1280  x  1024.  Figure  4  shows  the  IMPACT  HMD  symbology  format. 


The  HMD  was  used  as  a  “virtual  dome”  in  the  baseline  condition  and  only  displayed  the 
various  surface  and  air  threats  (Threat  “Circle”  in  Figure  4).  Flight  and  threat/target  cueing 
symbology  were  not  displayed  on  the  HMD  in  the  baseline  configuration.  Conversely,  the  HMD 
for  the  IMPACT  cockpit  did  display  flight  and  threat/target  cueing  symbology  as  well  as  the 
surface  and  air  threats  (everything  in  Figure  4). 

3.2.3  Directional  Audio 

The  Auditory  Localization  Cue  Synthesizer  (ALCS)  was  designed  by  Armstrong 
Laboratory  (AL)  in  conjunction  with  Systems  Research  Laboratory  (SRL).  This  audio  system 
provided  two  channels:  one  channel  for  azimuth  cueing  and  the  other  for  elevation  cueing.  The 
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head-related  transfer  functions  filter  sound  through  any  one  of  272  speaker  locations  (separated 
by  15°)  with  an  update  rate  of  50  Hz.  Because  it  was  necessary  to  communicate  with  the  pilot 
during  the  experiment,  one  channel  was  devoted  to  the  intercom,  thus  eliminating  elevation 
cueing  capability.  However,  the  decreased  demand  on  the  audio  processor,  as  well  as  the 
decrease  in  speaker  separation  within  the  head-related  transfer  function  (from  15°  to  1°),  allowed 
a  commensurate  increase  in  azimuth  cueing  resolution  and  greatly  enhanced  the  directional  audio 
capability. 

3.2.4  Head  Tracking  Device 

The  Flock  of  Birds  6-D  Multi-Receiver/Transmitter  Tracking  Device  was  used  to  measure 
the  pilot’s  head  X,  Y,  Z  position  coordinates  and  orientation  angles  and  provided  signals  to  both 
the  HMD  and  the  directional  audio  system.  The  signal  transfer  baud  rate  was  32K. 

3.2.5  Out-The-Window  Visual  System 

Simulated  FLIR  imagery  and  HUD  symbology  for  both  cockpits  were  generated  by  the 
Silicon  Graphics  system  and  were  displayed  on  a  Barco  Retrographics  801.  Once  the  pilot 
maneuvered  the  aircraft  to  put  the  target  within  the  simulated  HUD  FOV,  the  target  was  also 
displayed  on  the  Barco.  Refer  to  Figure  5  for  the  HUD  symbology. 
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Figure  5.  HUD  Symbology 


3.2.6  Experimenter’s  Station 

At  the  experimenter’s  station,  one  experimenter  could  monitor  the  simulation,  control  the 
cockpit  configuration,  and  start,  freeze,  and  restart  the  simulator  as  required.  This  experimenter 
also  controlled  the  sequencing  of  the  events  based  on  the  script,  route,  and  pilot  actions.  A 
second  experimenter,  stationed  immediately  behind  the  simulator,  observed  the  actions  of  the 
pilot,  provided  assistance  if  the  pilot  had  difficulty  with  the  simulator  or  the  helmet,  and  asked 
the  pilot  questions  at  the  end  of  each  trial.  Both  experimenters  and  the  subject  communicated  via 
intercom. 

3.2.7  Training  Material 

The  training  material  included  a  briefing  on  the  objectives  of  the  experiment,  a  description 
of  the  advanced  technologies,  a  briefing  on  the  baseline  and  IMPACT  displays  and  control  suites, 
and  a  description  of  the  experimental  procedures.  These  materials  were  used  to  familiarize  the 
subjects  with  the  experiment  procedures  and  simulator  operation. 
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3.2.8  Questionnaire  Material 


The  questionnaire  material  included  a  post-mission  data  probe,  an  out-brief  data  probe, 
and  a  methodology/procedures  questionnaire.  Examples  of  these  questionnaires  are  provided  in 
Appendix  C.  The  questionnaires  were  used  to  collect  subjective  data  on  the  advanced 
technologies,  their  implementation,  and  their  effects  on  performing  the  mission. 

3.3  Procedures 

The  test  procedures  consisted  of  the  training  and  testing  of  the  pilots,  the  experimental 
design,  and  the  collection  of  the  data.  Each  pilot  reported  to  the  laboratory  between  0800  and 
0830  for  introductory  briefings  and  training  on  experimental  procedures  and  equipment.  After 
the  training  session  was  completed,  each  pilot  was  given  a  practice  session  in  the  simulator  to 
provide  familiarity  with  the  mission  profile  and  procedures.  Data  collection,  which  consisted  of 
performance  data,  subjective  data,  and  questionnaire  data  followed  the  training  session.  The 
subjects  finished  the  day  at  approximately  1630  hours. 

3.3.1  Training 

The  training  program,  which  was  designed  to  help  all  the  pilots  understand  their  roles  and 
responsibilities  during  data  collection,  consisted  of  approximately  1 .5  hours  of  ground  training 
and  1.5  hours  of  simulator  familiarization  training.  The  ground  training  included  administrative 
items  such  as  the  facility  and  safety  considerations  and  an  introductory  briefing.  The  briefing 
included  a  description  of  the  following  information:  both  cockpit  configurations,  HUD  and 
HMD  symbology,  depictions  of  threats  and  targets,  mission  scenario,  and  test  procedures. 

All  pilots  were  introduced  to  the  Subjective  Workload  Assessment  Technique  (SWAT), 
which  is  a  method  of  collecting  workload  data  while  the  pilot  is  flying  the  simulator.  The 
introduction  to  SWAT  included  written  instructions,  verbal  instructions,  and  the  procedures  that 
the  pilots  should  use  during  the  data  collection  runs  to  relate  their  workload  levels  to  the 
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experimenters.1  The  training  session  concluded  with  simulator  familiarization,  in  which  the 
pilots  were  given  the  opportunity  to  practice  Threat  Acquisition  and  Weapon  Delivery  functions 
while  becoming  accustomed  to  flying  the  simulator. 

3.3.2  Testing 

At  the  completion  of  training,  each  pilot  participated  in  eight  data  collection  runs  of 
approximately  12  minutes  each.  The  scenario  was  a  night,  under  the  weather,  air  interdiction 
mission  attacking  a  SCUD  mobile  missile  launcher.  The  pilot  was  required  to  navigate  along  a 
pre-briefed,  pre-planned  route  and  to  perform  a  pop-up  weapon  delivery  using  a  simulated  FLIR 
system.  The  pilot  was  instructed  to  fly  the  HUD  flight  path  marker  in  the  manual  terrain 
following  (TF)  box  to  maintain  terrain  clearance  during  the  low-level  segment  of  the  mission. 
Manual  TF  was  flown  to  induce  additional  workload.  Figure  6  shows  a  graphic  of  the  air 
interdiction  (AI)  mission  scenario. 


1  Each  pilot  was  required  to  sort  a  set  of  27  cards.  The  cards  represented  the  various  dimensions  of  SWAT,  from 
lowest  to  highest  workload.  The  resulting  card  sort  was  then  subjected  to  a  statistical  technique  (conjoint 
measurement  technique)  to  develop  a  baseline  workload  scale,  from  0  (lowest  workload)  to  100  (highest  workload), 
for  each  subject  pilot  (Reid,  1989). 
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The  mission  was  divided  into  the  following  four  phases: 


1)  Medium  Altitude  Cruise  started  at  the  beginning  of  the  experiment,  ended  at  the  first 
turn  point  and  was  flown  at  10,000  feet  MSL,  at  480  knots. 

2)  Terrain  Following  Radar  (TFR)  Descent  started  at  the  first  turn  point  and  ended  at  the 
beginning  of  the  low-level  portion.  The  system  software  commanded  the  pilot  to  fly  45° 
of  bank  and  12°  of  dive  during  the  descent  while  increasing  the  aircraft  airspeed  to  540 
knots.  The  pilot  leveled  the  aircraft  at  300  feet  above  ground  level  (AGL). 

3)  High-Speed  TFR  Ingress  started  at  the  end  of  the  TFR  Descent  Phase  and  ended  at  the 
completion  of  the  turn  to  the  initial  point.  This  segment  was  flown  at  300  feet  AGL  and 
540  knots. 

4)  Weapons  Delivery/Attack  started  at  the  beginning  of  target  tracking  and  ended  at 
weapons  release.  The  “optimal”  attack  profile,  against  which  actual  pilot  performance 
was  compared,  required  10°  of  dive,  540  knots  groundspeed,  and  maintaining  an  attack 
course  specified  at  the  beginning  of  each  trial.  This  course  was  between  8,588  and 
4,100  feet  from  the  target. 

To  preclude  anticipatory  effects,  a  wide  range  of  values  was  established  for  the  location  of 
the  threats  and  ground  targets.  During  data  collection,  pilots  never  saw  the  same  threat  or  target 
location  more  than  once.  Refer  to  Appendix  D  for  values  that  were  used  during  testing. 

To  assess  pilot  threat  acquisition  and  weapon  delivery  performance  with  and  without  the 
advanced  technologies,  each  pilot  encountered  four  threats  along  the  route  and  then  attacked  one 
target.  Threats  and  targets  were  categorized  as  either  “easy”  or  “hard.”  The  reasons  for  the  two 
threat/target  types  were  operational  realism  and  the  avoidance  of  learning  effects. 

Threat  difficulty  was  based  on  the  position  of  the  threat  with  respect  to  the  aircraft  when  the 
threat  first  appeared.  Easy  threats  were  defined  as  being  in  the  pilot’s  forward  hemisphere  from 
0°  to  ±  90°  in  azimuth  and  from  0°  to  ±  50°  in  elevation.  The  hard  threats  were  defined  as  being 
in  the  pilot’s  aft  hemisphere,  from  90°  to  180°  in  azimuth  (left  and  right  hand  sides  of  the 
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cockpit)  and  from  50°  to  90°  in  elevation.  Threats  were  simulated  as  either  surface-to-air 
missiles  or  air-to-air  missiles  and  were  presented  on  the  HMD  as  stationary  small  green  circles. 

Easy  targets  were  located  within  10°  of  the  run-in  course  at  the  start  of  the  pop-up 
maneuver,  and  the  hard  targets  were  located  outside  the  HUD  FOV  (between  35°  and  45°  from 
run-in  course)  at  the  pop-up  point.  All  ground  targets  were  computer-generated  depictions  of 
SCUDs  and  appeared  on  the  simulated  LANTIRN  HUD  visual  scene.  Launcher  positions  were 
stationary  during  each  trial. 

In  the  baseline  condition,  threat  warnings  were  displayed  visually  on  the  tactical  electronic 
warfare  system  (TEWS)  and  aurally  through  the  radar  warning  receiver  system.  The  pilot 
obtained  general  azimuth  information  about  the  threats  by  looking  at  the  TEWS  display  and  was 
then  required  to  look  outside  the  cockpit  in  the  appropriate  direction  for  the  threat.  As  the  pilot’s 
head  turned  to  bring  the  threat  within  the  HMD  FOV,  the  threat  (a  small  green  circle)  was 
displayed  on  the  HMD.  Once  the  threat  was  acquired,  the  pilot  activated  the  chaff  dispense 
switch  on  the  control  stick,  simulating  countermeasures  activity.  At  this  point,  the  threat 
disappeared  and  a  time  stamp  was  collected. 

Visual  acquisition  of  threats  in  the  IMPACT  cockpit  was  accomplished  in  a  similar  manner, 
however,  the  pilot  received  additional  cueing  information.  Threat  warnings  were  displayed 
visually  on  the  TSD  and  aurally  (via  tones)  through  the  directional  audio  system.  Figure  7 
depicts  the  TSD  threat  warning  symbology.  In  addition,  threat  locator  symbology  was  provided 
through  the  HMD.  Because  of  the  additional  cueing  information,  the  IMPACT  team 
hypothesized  that  the  pilot  would  be  able  to  acquire  threats  with  less  maneuvering  of  the  aircraft. 
After  visually  acquiring  the  threat  circle,  the  pilot  dispensed  chaff  to  simulate  defeating  the 
threat. 
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To  test  pilot  flying  performance  during  the  attack  segment  with  and  without  the  advanced 
technologies,  each  pilot  ended  each  mission  by  flying  a  pop-up,  weapon  delivery  attack  profile 
using  the  FLIR  system  on  a  SCUD  missile  launcher.  The  pilot  flew  to  the  pre-briefed  pop-up 
point,  and  then  climbed  to  the  roll-in  altitude  for  a  10°  low-angle/low-drag  dive  bomb  delivery. 
The  target  was  displayed  on  the  Barco  projection  system  after  the  pilot  had  maneuvered  to  put 
the  target  within  the  simulated  FLIR/HUD  FOV.  In  the  baseline  configuration,  the  pilot  located 
the  target  and  accomplished  the  weapon  delivery  using  only  the  cueing  information  provided  by 
the  HUD  symbology.  In  the  IMPACT  cockpit,  the  pilot  used  both  the  HUD  and  the  off- 
boresight  cueing  symbology  on  the  HMD  to  locate  the  target  and  deliver  the  weapon.  In  both 
configurations,  once  the  target  was  located,  the  pilot  was  required  to  maneuver  so  that  the 
optimized  weapon  delivery  profile  could  be  accomplished. 
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Finally,  at  the  completion  of  each  trial  and  at  the  completion  of  the  testing  session, 
questionnaires  were  used  to  collect  subjective  data  on  the  cockpit  configurations  and  the 
implementation  of  the  advanced  technologies. 

3.4  Experimental  Design 

A  2  x  2  (Cockpit  Configuration  by  Threat  Difficulty)  blocked  counterbalanced  design  was 
used  in  this  experiment,  and  the  details  are  provided  in  Appendix  D.  Threat  difficulty  was 
balanced  so  that  the  overall  difficulty  for  an  entire  trial  was  approximately  the  same  for  all  trials. 
Two  easy  threats  and  two  hard  threats  appeared  in  every  trial,  and  the  presentation  order  was 
counterbalanced  across  missions  to  eliminate  systematic  learning  effects.  Finally,  the  difficulty 
of  ground  targets  was  blocked  across  pairs  of  successive  missions,  providing  an  equal  number  of 
easy  and  hard  target  engagements. 

3.5  Data  Collection 

Objective  measures  of  performance  were  used  to  determine  the  collective  contributions  of 
the  advanced  technologies  in  reducing  threat  visual  acquisition  times  and  flight  performance 
during  weapon  delivery.  Flight  path  tracking  accuracy  during  the  cruise,  low-level  and  attack 
portions  of  the  mission  was  collected.  Subjective  measurements  were  used  to  determine  the 
contributions  of  the  individual  technologies  for  accomplishing  the  AI  mission  tasks  and  to 
identify  baseline  versus  IMPACT  pilot  workload. 

Five  types  of  data  were  collected:  Measures  of  Effectiveness  (MOE),  Measures  of 
Workload  (MOW),  Measures  of  Situation  Awareness  (MSA),  Measures  of  Performance  (MOP), 
and  subjective  data.  MOEs  provided  a  relative  assessment  of  global  pilot- vehicle  system 
performance  across  experimental  conditions.  MSAs  provided  a  means  to  compare  relative  pilot 
situation  awareness  in  each  cockpit  configuration.  MOPs  provided  a  relative  measure  of  pilot 
performance  on  specific  tasks  for  various  experimental  conditions.  MOWs  assessed  mental 
workload  at  various  points  in  the  mission.  MOWs  can  be  correlated  with  MOPs  and  MOEs  to 
identify  performance  and  effectiveness  decrements  in  high-workload  conditions.  Finally, 
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subjective  data  included  pilot  ratings  and  comments  that  were  obtained  through  questionnaires. 
See  Table  1  for  a  summary  of  the  data  collected  from  each  mission  phase. 


Table  1.  Data  Collected  From  Each  Mission  Phase 


PARAMETER 

MISSION  PHASE 

Medium 
Altitude  Cruise 

TFR 

Descent 

High-Speed 
TFR  Ingress 

Weapon 

Delivery 

Ground/Airspeed  (knots) 

V 

V 

V 

Altitude  (feet) 

V 

Course  (degrees) 

V 

V 

V 

Dive  Angle  (degrees) 

V 

V 

TFR  Command  (milliradians) 

V 

Threat  Acquisition  Time  (sec.) 

V 

V 

V 

Workload  (SWAT) 

V 

V 

V 

V 

3.5.1  Measures  of  Effectiveness 

The  two  measures  of  effectiveness  in  this  experiment  were  threat  acquisition  time  and 
threat  acquisition  success  rate.  The  threat  acquisition  time  was  defined  as  the  interval  starting 
with  threat  onset  via  auditory  tone  and  visual  display  and  ending  with  the  pilot  dispensing  chaff 
using  the  paddle  switch  on  the  control  stick.  Pilots  were  instructed  to  stabilize  the  threat  symbol 
within  the  HMD  FOV  prior  to  dispensing  chaff.  If  not  acquired,  the  threat  would  disappear  after 
20  seconds.  The  “time-out”  value  was  based  on  the  experience  of  subject-matter  experts,  who 
stated  that  the  upper  bound  for  a  surface-to-air  or  air-to-air  threat  engagement  approaches  20 
seconds.  The  threat  acquisition  success  rate  was  defined  as  the  number  of  threats  acquired  within 
the  allotted  20  seconds  for  both  cockpit  configurations. 

3.5.2  Measures  of  Workload 

The  Subjective  Workload  Assessment  Technique  (SWAT)  was  used  to  collect  subjective 
pilot  workload  data  after  each  of  the  four  threats  and  after  weapon  delivery.  SWAT  assumes  that 
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workload  is  composed  of  three  dimensions:  time  stress,  mental  effort,  and  psychological  stress. 
Time  stress  refers  to  the  amount  of  time  available  to  an  operator  to  accomplish  a  task  and  is  rated 
on  a  3-point  scale  ranging  from  1  (often  have  spare  time)  to  3  (almost  never  have  spare  time). 
Mental  Effort  refers  to  the  amount  of  attention  or  concentration  that  is  required  to  perform  a  task 
and  is  rated  on  a  3 -point  scale  ranging  from  1  (very  little  conscious  effort)  to  3  (excessive  mental 
effort  and  concentration  is  required).  Psychological  Stress  refers  to  the  presence  of  confusion, 
frustration,  or  anxiety  associated  with  a  task  and  is  also  rated  on  a  3-point  scale  ranging  from  1 
(little  confusion,  risk,  frustration,  and/or  anxiety  exists  and  can  be  easily  accommodated),  to  3 
(high  to  very  intense  stress  due  to  confusion,  frustration,  or  anxiety)  (Reid,  1989). 

3.5.3  Measures  of  Situation  Awareness 

A  customized  Subjective  WORkload  Dominance  (SWORD)  form  was  used  to  gather  SA 
measures  and  to  compare  relative  pilot  SA  in  the  different  cockpit  configurations  and  tasks.  The 
SWORD  form  for  this  experiment  is  provided  in  Appendix  C  and  is  based  on  work  performed  by 
Vidulich  (1989).  The  relative  judgments  were  used  to  generate  ratings  for  each  cockpit 
configuration  and  task,  which  were  statistically  analyzed.  Pilots  completed  a  SWORD  rating 
form  during  the  post-experiment  questionnaire  session. 

3.5.4  Measures  of  Performance 

Relevant  measures  for  the  different  mission  phases  were  collected  to  determine  flight  path 
performance  during  threat  acquisition.  The  measures  included  aircraft  altitude,  course, 
groundspeed,  dive  angle,  and  terrain  following  (as  applicable)  for  the  Medium  Altitude  Cruise, 
TFR  Descent,  and  High-Speed  TFR  Ingress  Phases.  Deviations  from  the  required  flight  path 
were  collected  and  analyzed. 

The  aircraft’s  vertical  flight  path,  attack  course  axis,  and  groundspeed  were  measured  to 
determine  flight  performance  during  weapon  delivery.  Pilot  performance  was  compared  to  the 
pre-briefed  “optimal”  attack  flight  path,  and  deviations  from  the  optimal  path  were  collected  and 
analyzed.  The  flight  performance  results  are  provided  in  Section  4. 
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3.5.5  Subjective  Data 


The  pilots  were  administered  post-mission  interviews,  and  each  completed  a  pilot 
information  survey  and  questionnaire  at  the  end  of  the  experiment.  Questionnaires  and 
interviews  focused  on  cockpit  comparisons  and  the  influence  of  each  individual  advanced 
technology  on  overall  program  objectives  (see  Appendix  C). 
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4.  RESULTS 


The  results  of  this  experiment  address  the  objectives  stated  in  Section  1,  which  were  to 
evaluate  pilot  threat  acquisition  performance,  flight  performance,  workload,  and  SA  in  the 
baseline  and  IMPACT  cockpits,  as  well  as  to  determine  the  relative  strengths  and  weaknesses  of 
the  advanced  technologies.  Measures  of  effectiveness  for  the  configurations  include  the  time 
required  to  acquire  threats  and  the  threat  acquisition  success  rate.  Measures  of  pilot  flight 
performance  while  acquiring  threats  and  attacking  ground  targets  include  deviations  from 
planned  altitude,  course,  and  groundspeed.  Pilot  workload  and  situation  awareness  associated 
with  the  threat  acquisition  and  weapon  delivery  functions  of  the  missions  for  both  cockpit 
configurations  were  collected  and  are  provided  in  this  section.  Finally,  the  subjects’  opinions  of 
the  strengths  and  weaknesses  as  to  how  the  advanced  technologies  were  implemented  in  this 
experiment  are  summarized.  Statistical  results  are  provided  in  Appendix  F. 

4.1  Measures  of  Effectiveness 

To  reiterate,  the  measures  of  effectiveness  included  the  threat  acquisition  time  and  the  threat 
acquisition  success  rate.  Threat  acquisition  time  was  the  period  of  time  from  the  initial 
appearance  of  the  threat  until  the  pilot  visually  acquired  the  threat  and  dispensed  chaff.  The 
threat  acquisition  success  rate  was  the  number  of  threats  acquired  divided  by  the  number  of 
threats  presented. 

4.1.1  Threat  Acquisition  Time 

A  2  x  2  (Cockpit  Configuration  by  Threat  Difficulty)  repeated  measures  ANalysis  Of 
VAriance  (ANOVA)  was  performed  on  the  threat  acquisition  time  data.  The  threat  acquisition 
time  was  significantly  shorter  for  the  IMPACT  cockpit  than  for  the  baseline  in  both  the  easy 
threat  condition  and  the  hard  threat  condition  (see  Figure  8). 

The  threat  acquisition  time  was  significantly  shorter  with  the  IMPACT  cockpit  (8.43  sec.) 
than  for  the  baseline  (12.95  sec.).  Also,  threat  acquisition  was  significantly  shorter  for  the  easy 
threats  (8.28  sec.)  than  for  the  hard  threats  (13.10  sec.). 
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4.1.2  Threat  Acquisition  Success  Rate 

A  2  x  2  (Cockpit  Configuration  by  Threat  Difficulty)  repeated  measures  ANOVA  was 
performed  on  the  threat  acquisition  success  rate  data.  As  shown  in  Figure  9,  approximately  20% 
more  threats  were  acquired  with  the  IMPACT  cockpit  than  with  the  baseline  throughout  the 

entire  mission. 


Figure  9.  Mean  Threat  Acquisition  Success  Rate 


28 


A  significant  main  effect  for  Threat  Difficulty  was  found  as  a  tunction  of  the  threat 
acquisition  success  rate.  The  success  rate  was  significantly  higher  for  easy  threats  (91.60%)  than 
for  hard  threats  (83.91%). 

4.2  Measures  of  Workload 

A  2  x  2  (Cockpit  Configuration  by  Threat  Difficulty)  repeated  measures  ANOVA  was 
performed  on  the  SWAT  data.  The  SWAT  scores  were  significantly  lower  for  the  IMPACT 
cockpit  than  for  the  baseline  in  both  the  easy  and  hard  threat  conditions  (see  Figure  10). 


Figure  10.  Mean  SWAT  Scores 


A  significant  main  effect  for  Cockpit  Configuration  was  found  as  a  function  of  SWAT 
scores.  Workload  was  significantly  lower  for  the  IMPACT  cockpit  (Mean  SWAT  score  =  33.20) 
than  for  the  baseline  (Mean  SWAT  score  =  50.78). 

A  significant  main  effect  for  Threat  Difficulty  was  found  as  a  function  of  SWAT  scores. 
SWAT  scores  were  significantly  lower  for  easy  threats  (Mean  SWAT  score  =  34.23)  versus  hard 
threats  (Mean  SWAT  score  =  49.75). 
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For  the  Weapon  Delivery  Phase,  a  separate  2x2  (Cockpit  Configuration  by  Target 
Difficulty)  repeated  measures  ANOVA  was  performed  on  the  SWAT  data.  No  significant 
differences  were  found  in  SWAT  scores  regarding  Target  Difficulty  or  Cockpit  Configuration. 

4.3  Measures  of  Situation  Awareness 

A  Subjective  WORkload  Dominance  (SWORD)  form  was  given  to  the  pilots  to  evaluate 
situation  awareness  for  the  Mission  Functions.  Mission  Functions  included  threat  acquisition, 
target  acquisition,  and  fly/navigate.  A  2  x  3  (Cockpit  Configuration  by  Mission  Function) 
repeated  measures  ANOVA  was  performed  on  the  SWORD  data  and  the  results  are  shown  in 
Figure  1 1. 

A  significant  Cockpit  Configuration  by  Mission  Function  interaction  was  discovered.  SA 
was  rated  higher  for  the  IMPACT  cockpit  than  for  the  baseline  during  the  Threat  Acquisition  and 
Target  Acquisition  functions.  There  was  no  statistically  significant  difference  between  cockpits 
for  SA  during  the  Fly/Navigate  function. 
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Figure  11.  Mean  SWORD  Situation  Awareness  Ratings 

4.4  Measures  of  Performance 

Flight  performance  parameters,  such  as  altitude,  heading,  and  airspeed  were  collected  and 
analyzed  for  the  threat  acquisition  and  weapon  delivery  functions.  Deviations  from  the  planned 
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altitude,  heading,  and  airspeed  were  analyzed  for  each  Cockpit  Configuration  and  Threat 
Difficulty  level. 

4.4.1  Flight  Performance  During  Threat  Acquisition 

All  flight  path  tracking  analyses  were  conducted  using  a  2  x  2  (Cockpit  Configuration  by 
Threat  Difficulty)  repeated  measures  ANOVA.  Separate  analyses  were  conducted  based  on  the 
Mission  Phase  (Medium  Altitude  Cruise,  TFR  Descent,  and  TFR  Ingress)  because  each  phase 
had  different  MOPs  and  different  criteria.  Data  collected  only  during  threat  engagements  were 
analyzed. 

During  threat  acquisition,  deviations  from  planned  altitude,  course,  and  groundspeed  were 
analyzed.  A  significant  Configuration  by  Threat  Difficulty  interaction  was  found.  Altitude 
deviation  was  significantly  lower  for  the  IMPACT  cockpit  (107.95  ft.)  compared  to  the  baseline 
(285.68  ft.)  in  the  easy  threat  condition.  No  significant  effects  were  found  between  the  IMPACT 
cockpit  and  the  baseline  in  the  hard  threat  condition. 

A  significant  main  effect  for  Cockpit  Configuration  as  a  function  of  course  deviation  was 
found.  Regardless  of  threat  difficulty,  pilots  maintained  course  significantly  better  with  the 
IMPACT  cockpit  versus  the  baseline  cockpit  (see  Figure  12). 


0£  Baseline  IMPACT 

Cockpit  Configuration 


Figure  12.  Mean  RMS  Course  Deviation  During  Medium  Altitude  Cruise 
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Finally,  for  groundspeed,  no  significant  differences  were  found  between  the  cockpit 
configurations. 

Significant  main  effects  were  found,  however,  for  Threat  Difficulty  as  a  function  of  both 
course  deviation  and  groundspeed  deviation.  Smaller  course  deviations  were  found  in  the  easy 
threat  condition  (6.85°)  than  in  the  hard  threat  condition  (11.58°).  Smaller  groundspeed 
deviations  were  found  in  the  easy  threat  condition  (18.50  kts.)  than  in  the  hard  threat  condition 
(49.99  kts.).  No  other  interaction  or  main  effects  were  discovered. 

Deviation  from  planned  dive  angle  was  analyzed  in  the  TFR  Descent  Phase.  No 
interaction  effects  or  main  effects  were  found. 

Deviations  from  planned  course,  groundspeed,  and  terrain  following  were  analyzed  in  the 
TFR  Ingress  Phase.  Significant  main  effects  for  Cockpit  Configuration  were  found  for  course 
deviation  and  TFR  deviation  with  the  IMPACT  cockpit  demonstrating  significantly  less 
deviations  compared  to  the  baseline.  No  significant  differences  were  found  between  cockpit 
configurations  for  groundspeed  deviation.  Significant  main  effects  were  also  found  for  Threat 
Difficulty  as  a  function  of  course  deviation,  groundspeed  deviation,  and  TFR  deviation  with  easy 
threats  resulting  in  smaller  deviations  than  hard  threats. 

4.4.2  Flight  Performance  During  Weapon  Delivery 

Weapon  Delivery  analysis  was  conducted  using  a  2  x  2  (Cockpit  Configuration  by  Target 
Difficulty)  repeated  measures  ANOVA.  Deviations  from  planned  vertical  flight,  attack  course, 
and  groundspeed  were  analyzed  in  this  phase. 

A  significant  main  effect  for  Target  Difficulty  was  found  only  for  vertical  flight  deviation. 
The  pilots  had  significantly  less  vertical  flight  deviation  in  the  easy  target  condition  (315.42  ft.) 
than  in  the  hard  target  condition  (503.28  ft.).  No  other  main  effects  or  interactions  were  found 
for  attack  course  deviation  or  groundspeed  deviation.  In  addition,  no  significant  differences  were 
found  between  cockpit  configurations  for  any  of  the  flight  performance  variables. 
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4.5  Subjective  Data 


Frequencies  and  mean  ratings  were  calculated  for  all  questionnaire  items.  Complete 
responses  to  open-ended  questions  and  other  pilot  comments  are  included  in  Appendix  E.  Both 
post-mission  and  post-experiment  questionnaires  were  used  to  address  the  overall  contributions 
of  each  technology  and  the  specific  individual  contributions  of  each  technology  element. 

4.5.1  Threat  Acquisition  Effectiveness 

The  pilots  rated  threat  acquisition  easier  in  the  IMPACT  cockpit  than  in  the  baseline. 

Because  of  the  HMD  threat  cueing  capability,  1 6  out  of  1 8  pilots  specifically  commented 
that  IMPACT  was  a  vast  improvement  over  the  baseline  for  threat  acquisition.  Conversely,  six 
out  of  1 8  pilots  commented  that  threat  acquisition  in  the  baseline  was  more  difficult,  mainly  due 
to  the  lack  of  threat  elevation  cueing. 

Threat  acquisition  was  composed  of  three  elements:  determining  azimuth,  determining 
elevation,  and  determining  range.  Figure  13  shows  pilot  ratings  for  the  overall  cockpit 
effectiveness  for  each  of  these  tasks. 


Figure  13.  Average  Cockpit  Effectiveness  Ratings  for  Determining  Threat  Location 
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The  pilots  were  also  asked  to  assess  the  contributions  of  each  technology  to  the  threat 
acquisition  elements  (azimuth,  elevation,  range).  According  to  their  responses,  Figure  14  shows 


Figure  14.  Contributions  of  Technologies  to  Threat  Localization 


that  the  strongest  contribution  of  the  HMD  was  in  its  ability  to  aid  the  pilot  in  determining  threat 
azimuth  and  elevation.  The  TSD  made  its  greatest  contributions  in  aiding  threat  azimuth  and 
range  determination,  and  directional  audio  made  a  moderate  contribution  in  threat  azimuth 
determination. 

4.5.2  Weapon  Delivery  Effectiveness 

Target  Acquisition  was  rated  slightly  easier  in  IMPACT  than  in  the  baseline  cockpit.  The 
following  pilot  comment  provides  a  possible  reason  why  this  rating  was  given:  “I  used  (the) 
helmet  to  see  when  to  roll  out .  . .  (I)  had  more  time  than  I  would  have  had  otherwise.”  Figure  15 
depicts  cockpit  effectiveness  for  the  Target  Acquisition  and  Weapon  Delivery  functions.  The 
IMPACT  configuration  was  rated  more  effective  than  the  baseline  for  the  Target  Location  and 
Weapons  Delivery  functions. 
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Figure  15.  Average  Cockpit  Effectiveness  Ratings 
for  Locating  Target  and  Delivering  Weapon 


4.5.3  Aircraft  Control 

The  pilots  were  asked  to  assess  the  cockpits  for  flying  difficulty.  They  reported  that  they 
found  both  cockpits  moderately  easy  to  fly,  with  IMPACT  being  slightly  easier. 

When  the  pilots  were  asked  for  cockpit  effectiveness  ratings  on  the  post-experiment 
questionnaire,  they  rated  the  IMPACT  cockpit  slightly  more  effective  than  the  baseline.  Figure 
16  shows  the  results. 


Figure  16.  Average  Cockpit  Effectiveness  Ratings  (Flying) 
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The  contributions  of  the  three  advanced  technologies  to  situation  awareness  are  shown  in 
Figure  17,  with  the  TSD  making  the  largest  contribution.  Six  out  of  18  pilots  specifically 
identified  the  IMPACT  TSD  as  providing  the  greatest  level  of  situation  awareness. 


Figure  1 7.  Contributions  of  Technologies  to  Maintaining  Situation  Awareness 

4.5.4  Hardware  and  Design  Characteristics 

The  design  characteristics  of  the  three  advanced  technologies  were  examined  to  determine 
pilot  acceptance.  Figure  1 8  shows  the  acceptability  ratings.  The  TSD  was  the  closest  of  the 
three  to  being  rated  “completely  acceptable”,  with  the  directional  audio  rated  moderately 
acceptable”.  Some  pilots  stated  that  the  TSD  was  better  than  a  wingman  for  conveying  whether 
they  are  being  targeted  by  a  threat,  and  one  said  that  it  was  ‘  ...the  best  SA  device  ever. 


Figure  18.  Average  Acceptability  Ratings  of  Design  Characteristics 
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5.  DISCUSSION 


In  this  section,  the  effects  of  the  experimental  results  and  their  relationships  to  the  following 
four  objectives  are  addressed:  threat  acquisition  performance,  pilot  flight  performance  during 
weapon  delivery,  pilot  workload  and  situation  awareness,  and  the  strengths  and  weaknesses  of 
the  cockpit  mechanization. 

5.1  Threat  Acquisition  Performance 

The  first  objective  of  this  experiment  was  to  evaluate  threat  acquisition  performance  in  both 
the  baseline  cockpit  and  the  IMPACT  cockpit  configuration.  Specific  points  of  interest  were  the 
threat  acquisition  time  and  the  threat  acquisition  success  rate.  Statistical  analysis  showed  that 
overall,  the  IMPACT  cockpit  elicited  significantly  faster  acquisition  times  and  higher  success 
rates  than  the  baseline.  The  large  TSD,  the  HMD,  and  directional  audio  gave  the  pilot  a  more 
detailed  depiction  of  threat  location.  These  technologies  allowed  the  pilot  to  acquire  the  threat 
quicker  and  with  less  aircraft  maneuvering,  which  resulted  in  a  higher  success  rate,  significantly 
lower  workload,  and  better  SA. 

Regarding  threat  acquisition,  the  pilots  greatly  favored  the  implementation  of  the  HMD. 
Threat  cueing  symbology  was  rated  particularly  high  for  determining  threat  azimuth  and 
elevation.  The  TSD  was  given  high  acceptability  ratings  and  provided  the  pilot  with  additional 
threat  range  information  as  well  as  awareness  of  threat  type  (air-to-air  or  air-to-ground).  Finally, 
several  of  the  pilots  commented  favorably  concerning  the  directional  audio  technology.  Even 
though  it  received  the  lowest  rating  of  all  the  advanced  technologies  studied,  directional  audio 
appeared  to  provide  initial  threat  azimuth  information  to  the  pilots  who  used  it.  Implementation 
of  a  true  3-D  audio  system  may  provide  more  compelling  results. 

5.2  Flight  Performance  During  Threat  Acquisition 

Associated  with  the  assessment  of  threat  acquisition  time  (the  primary  objective  of  this 
experiment)  was  the  evaluation  of  flight  performance  during  threat  acquisition.  In  the  IMPACT 
cockpit,  the  pilots  were  significantly  more  precise  in  maintaining  the  specified  course  and  terrain 
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following  parameters  while  acquiring  threats.  This  improvement  in  performance  was  true 
regardless  of  threat  difficulty  and  was  most  likely  due  to  the  tendency  of  the  pilots  to  maneuver 
the  IMPACT  cockpit  less  than  the  baseline  (because  of  the  additional  cueing  capability  in  the 
IMPACT  configuration).  When  flying  the  baseline  cockpit,  pilots  recognized  the  system 
limitations  and  were  very  methodical  in  locating  threats.  They  typically  maneuvered  to  place  the 
threats  in  their  3  or  9  o’clock  position  (resulting  in  large  course  deviations),  rolled  the  aircraft 
wings  level,  and  then  began  to  search  for  the  threat  in  elevation. 

One  concern  is  the  major  deviations  in  altitude  and  groundspeed  that  were  observed  with  the 
IMPACT  cockpit  during  the  Medium  Altitude  Cruise  Phase.  Although  not  statistically 
significant,  preliminary  evidence  may  exist  that  could  indicate  a  degradation  of  cockpit  cross¬ 
checking  during  a  high  threat  situation  at  medium  to  high  altitudes.  One  cause  may  be  the  added 
capability  of  the  HMD,  allowing  the  pilot  to  immediately  go  “heads-up”  using  the  cueing 
symbology  to  locate  threats.  Naturally,  the  more  time  it  takes  to  locate  a  threat,  the  less  time  the 
pilot  spends  “heads-down”  while  cross-checking  the  instruments.  Therefore,  additional  HMD 
flight  symbology  or  training  may  be  necessary  to  diminish  the  possibility  of  pilots  losing 
awareness  of  the  aircraft  attitude. 

5.3  Flight  Performance  During  Weapon  Delivery 

The  second  objective  of  this  experiment  was  to  evaluate  the  pilots’  flight  performance  while 
attacking  ground  targets.  Contrary  to  the  team’s  hypothesis,  the  IMPACT  configuration  showed 
no  improvement  over  the  baseline  for  weapon  delivery.  Even  though  the  expanded  TSD  and  the 
target  locator  line  provided  in  the  HMD  promoted  easier  target  acquisition  with  IMPACT,  which 
is  supported  by  post-mission  ratings  provided  by  the  pilots,  flight  performance  during  the 
Weapon  Delivery  Phase  was  consistent  across  both  configurations. 

The  lack  of  statistically  significant  results  during  weapon  delivery  may  be  attributed  to  the 
difficulty  of  the  maneuver.  When  a  pilot  is  trained  to  fly  fighter  aircraft,  pop-up  weapons 
deliveries  are  among  the  last  to  be  taught.  These  maneuvers  are  normally  accomplished  only  in 
the  daytime.  Although,  some  F-15E  and  F-16  pilots  are  certified  to  accomplish  this  at  night 
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using  the  LANTIRN  system,  they  only  attack  targets  that  are  within  the  LANTIRN  FOV  during 
the  maneuver.  The  inherent  difficulty  of  the  maneuver  combined  with  trying  to  accomplish  it  in 
a  simulator  without  the  cues  normally  associated  with  flying  an  aircraft  (e.g.,  g-forces  and 
vibrations)  made  this  task  very  challenging.  Simulation  tests  were  conducted  to  determine  if 
advanced  technologies  could  better  support  the  pilot  in  performing  such  maneuvers.  However, 
once  the  pop-up  maneuver  was  initiated  in  the  advanced  configuration,  pilots  seemed  to 
concentrate  on  the  HUD.  The  HUD  format  was  the  same  for  both  the  baseline  and  IMPACT 
configurations,  which  probably  negated  potential  differences. 

Even  though  considerable  training  was  conducted  in  the  use  of  the  HMD  during  roll-in, 
some  of  the  pilots’  comments  indicated  they  did  not  begin  to  exploit  the  capability  until  later  data 
runs.  More  extensive  training  (not  possible  under  the  time  constraints  of  this  experiment)  may 
have  resulted  in  more  effective  use  of  the  HMD  symbology  for  this  task.  Another  factor  that 
could  have  affected  results  was  the  lack  of  a  smooth  transition  of  the  target  designator  diamond 
from  the  HMD  to  the  HUD.  With  a  “seamless”  transition  of  the  target  diamond  (as  would  be 
possible  with  an  HMD-projected  “virtual”  HUD)  a  more  intuitive  use  of  the  symbology  could 
result. 

5.4  Pilot  Workload  and  Situation  Awareness 

The  third  objective  of  this  experiment  was  to  evaluate  pilot  workload  and  SA  in  both  the 
IMPACT  and  baseline  cockpits.  Statistical  analysis  showed  that  the  IMPACT  cockpit  elicited 
significantly  lower  workload  than  the  baseline  with  the  exception  of  the  weapon  deliveiy 
function.  No  significant  workload  differences  were  found  during  weapon  delivery  primarily 
because  the  inherent  difficulty  of  the  weapon  delivery  maneuver,  combined  with  trying  to 
accomplish  the  maneuver  in  a  simulator  without  the  cues  normally  associated  with  flying  an 
aircraft  (e.g.,  g-forces  and  vibrations)  made  this  task  very  challenging.  Once  the  pop-up 
maneuver  was  initiated  in  the  advanced  configuration,  pilots  seemed  to  concentrate  on  the  HUD, 
and  because  the  HUD  format  was  the  same  for  both  the  baseline  and  IMPACT  configurations, 
potential  workload  differences  were  minimized. 
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Statistical  analysis  results  indicated  a  significant  increase  in  SA  with  the  IMPACT  cockpit 
relative  to  the  baseline  (with  the  only  exception  being  navigation).  Factors  contributing  to  this 
result  can  be  attributed  to  the  added  threat  location  capabilities  provided  by  IMP  ACT  s  advanced 
technologies.  Specifically,  pilots  found  it  easier  to  acquire  threats  using  the  HMD  and  TSD 
cueing  symbology,  and  these  technologies  appear  to  be  the  main  cause  of  the  decrease  in 
workload.  According  to  pilot  comments  the  main  contributors  to  increased  SA  were  the  larger, 
more  precise  TSD  range  scale,  the  threat/target  symbology,  and  navigation  symbology. 

5.5  Cockpit  Mechanization 

The  forth  objective  of  this  experiment  was  to  identify  relative  strengths  and  weaknesses  of 
the  IMPACT  technologies.  Overall,  the  pilots  rated  them  very  highly,  with  the  large  TSD  rated 
the  highest.  Concerns  regarding  the  HMD  centered  more  around  the  hardware  than  the  display 
design.  Since  this  HMD  was  a  prototype  system,  the  helmet  was  not  as  comfortable  nor  was  it  as 
adjustable  as  it  would  be  in  an  operational  environment.  Image  quality  was  another  issue  raising 
concern.  In  fact,  before  the  HMD  could  be  used  in  an  operational  environment,  several  issues 
would  have  to  be  addressed,  including:  helmet  center-of-gravity  (because  of  the  added  weight  of 
the  display  combiners),  CRT  alignment,  focus,  brightness  under  field  conditions,  suitable 
adjustment  devices  that  provide  sufficient  setting  feedback,  fine-tuning  capabilities,  and  ease  of 
use  on  the  ground  and  during  flight. 

The  only  major  HMD  symbology  design  concern  raised  by  the  pilots  was  the  conflict 
between  the  threat  locator  line  and  the  horizon  line.  These  two  solid  lines  would  overlap  if  the 
threat  was  located  at  the  same  altitude  as  the  aircraft.  This  issue  will  require  further  study,  but 
may  become  less  apparent  in  a  fully  dynamic  threat  environment  (threats  were  simulated  in  fixed 
locations  for  this  experiment). 

The  TSD  was  the  most  conventional  of  the  three  advanced  technologies  and,  as  such,  was 
well  received  by  the  pilots.  Suggested  improvements  focused  on  the  separation  of  information. 
During  several  threat  engagements,  threat  symbology  overlapped  with  navigation  symbology 
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making  both  sets  difficult  to  read.  Better  prioritization  of  displayed  information  should  remedy 
this  problem. 

Finally,  directional  audio  capabilities  must  continue  to  be  developed  and  evaluated  to  obtain 
a  more  accurate  depiction  of  its  potential  contribution. 
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6.  CONCLUSIONS 


This  experiment  was  successful  in  evaluating  three  specific  advanced  technologies:  HMD, 
large  TSD,  and  directional  audio.  The  integration  of  these  technologies  into  the  IMPACT 
cockpit  provided  significant  benefits  over  the  baseline  configuration.  With  regard  to  threat 
acquisition,  pilots  were  successful  in  performing  functions  that  are  currently  augmented  by  the 
WSO  without  increasing  workload  or  decreasing  situation  awareness.  Continued  effort  is 
required,  both  in  the  development  of  technology  and  simulation  capability,  to  obtain  a  more 
accurate  indication  of  performance  concerning  weapon  delivery  in  the  IMPACT  cockpit.  These 
efforts  are  very  important  steps  in  developing  a  single-seat,  multi-role  fighter  aircraft  required  to 
perform  precision  attack  at  night  and  in  adverse  weather. 

Further  development  of  the  HMD,  large  TSD,  and  directional  audio,  as  well  as  the 
integration  of  other  advanced  technologies,  are  the  objectives  of  future  IMPACT  experiments. 
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7.  RECOMMENDATIONS 


Further  development  of  the  HMD,  TSD,  directional  audio,  and  additional  integration  of 
advanced  target  detection  and  designation  technologies  is  necessary  to  accomplish  the  goal  of 
defining  single-seat,  multi-role  fighter  cockpit  design  requirements. 

•  HMD 

Specifically,  HMD  symbology  requires  evaluation  under  dynamic  threat  conditions. 
Additional  symbology  may  be  necessary  for  pilots  to  control  the  aircraft  more  effectively  while 
directing  attention  outside  the  aircraft.  This  is  especially  true  during  manual  terrain  following 
conditions.  Also,  more  HMD  capability  must  be  integrated  for  weapon  delivery  functions. 

•  TSD 

Refinement  of  the  TSD  is  necessary  to  prioritize  information.  For  example,  threat 
information  should  take  precedence  over  navigation  information  unless  otherwise  requested  by 
the  pilot.  This  type  of  information  prioritization  would  reduce  display  clutter. 

•  Audio 

Additional  hardware  capabilities  are  required  to  properly  implement  3-D  audio  technology. 
Further  evaluation  is  necessary  under  more  realistic  operational  environments  to  completely 
understand  and  utilize  the  potential  benefits  of  advanced  audio  displays. 

Finally,  further  evaluation  of  advanced  technologies  is  required  to  determine  their  effects  on 
pilot  workload  and  SA  in  the  absence  of  the  WSO.  Specifically,  additional  research  is  necessary 
to  measure  the  ability  of  a  single  pilot,  with  the  support  of  advanced  technologies,  to  execute 
functions  currently  performed  by  the  WSO  during  the  weapon  delivery  phase  of  an  interdiction 
mission.  These  functions  include  controlling  the  air-to-ground  radar  and  patch  map  designation 
of  ground  targets. 
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APPENDIX  A 


SIMULATION  PROJECT  MODIFICATION  REQUESTS  (SPMRs) 


INTRODUCTION 


These  Simulation  Project  Modification  Requests  (SPMRs)  define  the  simulator  configuration 
used  in  IMPACT  Advanced  Technology  Experiment  #  2.  The  SPMRs  are  grouped  into  three 
sections: 


•  Baseline  Cockpit  Requirements 

•  IMPACT  Cockpit  Requirements 

•  Common  Simulation  Requirements 

Throughout  the  SPMRs  (Appendix  A),  Questionnaires  and  Data  Collection  Forms  (Appendix  C), 
and  Experimental  Design  (Appendix  D)  all  references  to  the  “F-15E”  have  been  changed  to  the 
“Baseline”  cockpit.  This  reduces  potential  confusion  that  the  Baseline  cockpit  simulated  in  the 
experiment,  which  had  the  F-15E  as  its  “model”,  had  the  same  level  of  functionality  as  a  high- 
fidelity  F-15E  simulation. 

The  SPMRs  describe  the  control  and  display  configurations  that  were  originally  envisioned  for 
the  baseline  and  IMPACT  cockpits.  Numerous  modifications  were  made  to  the  controls  and 
displays  while  refining  the  experimental  test  plan,  implementing  the  designs,  and  performing 
pre-experimental  checkouts  prior  to  the  study. 

Several  controls  and  displays  developed  for  the  IMPACT  Program  were  not  implemented  in  this 
experiment,  and  are  annotated  in  the  SPMR  Table  of  Contents  accordingly. 

To  document  the  changes  and  record  the  design  team’s  rationale  for  the  changes,  a 
REMARKS/LESSONS  LEARNED  section  has  been  added  to  each  SPMR. 
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TABLE  OF  CONTENTS 


SECTION  1:  BASELINE  COCKPIT  REQUIREMENTS 

SPMB  TITLE 

1 .0  Cockpit  Layout 

1 . 1  Stick  and  Throttles 

1 .2  Pre-Programmed  Display  Rotation  (not  implemented  for  this  study) 

1 .3  Head-Up  Display  (HUD) 

1 .4  Up-Front  Control  (UFC) 

1 .5  Air-to-Air  (A/A)  Radar  (not  implemented  for  this  study) 

1 .6  Air-to-Ground  (A/G)  Radar  (not  implemented  for  this  study) 

1 .7  Targeting  Pod  (not  implemented  for  this  study) 

1 .8  Tactical  Electronic  Warfare  System  (TEWS) 

1 .9  Terrain-Following  (TF)  Radar  Display 

1.10  Tactical  Situation  Display  (TSD) 

1.11  Programmable  Armament  Control  Set  (PACS)  (not  implemented  for  this  study) 

1.12  Attitude  Director  Indicator  (ADI) 

1.13  Horizontal  Situation  Indicator  (HSI)  (not  implemented  for  this  study) 

1.14  Engine  Monitor  Display 
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SPMR 

2.0 

2.1 

2.2 

2.3 

2.4 

2.5 

2.6 

2.7 

2.8 

2.9 

2.10 
2.11 
2.12 

2.13 

2.14 

2.15 

2.16 


SECTION  2:  IMPACT  COCKPIT  REQUIREMENTS 
TITLE 

Cockpit  Layout 
IMPACT  Stick  and  Throttles 

Display  Cursor  Control  (not  implemented  for  this  study) 

Head-Up  Display  (HUD) 

Up-Front  Control  (UFC) 

Air-to-Air  (A/A)  Radar  (not  implemented  for  this  study) 

Air-to-Ground  (A/G)  Radar  (not  implemented  for  this  study) 
Sensor/AGM  Video  (not  implemented  for  this  study) 

3-D  Audio 

Terrain-Following  (TF)  Radar 
Tactical  Situation  Display  (TSD) 

Weapons/Countermeasures  (WPNS/CM)  Display 
Attitude  Director  Indicator  (ADI) 

Horizontal  Situation  Indicator  (HSI)  (not  implemented  for  this  study) 
Engine/Fuel  Display 
Helmet-Mounted  Display  (HMD) 

Emergency  Procedures  (EP)  Checklists  (not  implemented  for  this  study) 
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SECTION  3:  COMMON  SIMULATION  REQUIREMENTS 


SPMR 

TITLE 

3.0 

HUD  Display  on  Barco  Projection  Screen 

3.1 

Forward-Looking  Infra-Red  (FLIR)  Display 

3.2 

Out-the-Window  Display 

3.3 

Feature  Analysis  Display 

3.4 

Targets 

3.5 

Threats 

SPMR  1.0 


COCKPIT  LAYOUT 

The  baseline  front  cockpit  was  modeled  after  the  F-15E  and  was  characterized  by  three 
multipurpose  displays  (MPDs),  an  up-front  control  (UFC),  and  a  wide  field-of-view  heads-up 
display  (HUD),  as  shown  in  Figure  1.0.1.  Note  the  numbering  scheme  used  for  the  pushbuttons 
on  the  MPDs  (PB  1  through  PB  20,  counterclockwise  starting  from  the  upper  left  comer). 


REMARKS/LESSONS  LEARNED 

The  UFC  and  MPDs  were  non-interactive  for  Advanced  Technology  Experiment  #1.  Selected 
data  displays  were  dynamic,  but  the  pilot  could  not  change  formats,  enter  data,  or  select 
operating  modes.  The  displays  were  presented  on  a  large  color  cathode  ray  tube  (CRT). 


SPMR  1.1 


BASELINE  STICK  AND  THROTTLES 


STICK 

An  F-15E  stick  grip  will  be  used  in  the  baseline  cockpit  for  Advanced  Technology  Experiment  # 
2.  The  stick  will  be  folly  operational  for  control  of  aircraft  pitch  and  roll.  The  switches  on  the 
stick  grip  that  must  function  for  the  first  demonstration  are  the  trim  switch  and  the  castle  switch. 
(See  Figure  1.1.1).  The  trim  switch  is  used  for  aircraft  trim  and  countermeasures  dispense,  and 
the  castle  switch  is  used  for  pre-programmed  display  rotation. 


REMARKS/LESSONS  LEARNED 

The  grip  for  the  side-stick  used  in  the  actual  simulation  was  slightly  different  from  the 
illustration.  Since  the  pilot  was  not  required  to  change  the  display  formats  for  Advanced 
Technology  Experiment  #  2,  the  castle  switch  function  was  not  implemented.  After  pre- 
experimental  sessions  revealed  minor  problems  with  planned  methods  for  countermeasures 
dispense  function,  the  design  team  decided  to  use  a  paddle  switch  (not  illustrated),  located  at  the 
front  base  of  the  stick,  to  execute  the  function  (instead  of  the  trim  switch). 
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THROTTLES 

The  throttles  must  be  capable  of  controlling  engine  rpm,  operating  speedbrakes,  and  dispensing 
countermeasures. 

The  speedbrake  switch  has  three  positions:  FORWARD,  CENTER,  and  AFT. 

•  FORWARD  Retracts  speedbrakes 

•  CENTER  Holds  speedbrakes  in  current  position 

•  AFT  Extends  speedbrakes 

The  CMD  switch  has  two  positions:  UP  and  DOWN 

•  UP  Manual  2  dispense 

•  DOWN  Manual  1  dispense 


REMARKS/LESSONS  LEARNED 

The  throttles  in  the  simulator  used  for  Advanced  Technology  Experiment  #  2  were  slightly 
different  in  configuration  from  the  F-15E,  but  had  functionally  equivalent  switches.  The  original 
plan  was  to  use  the  “Pinky”  switch  to  dispense  chaff  in  the  experiment.  However,  due  to  the 
somewhat  awkward  motion  required  to  operate  this  switch  and  the  need  to  obtain  precise  data 
concerning  the  timing  of  chaff  dispensing,  the  team  decided  to  use  the  paddle  switch  on  the 
control  stick. 


SPMR  1.3 


HEAD-UP  DISPLAY  (HUD) 


INTRODUCTION 

F-15E  HUD  symbology  will  be  used  for  IMPACT  Advanced  Technology  Experiment  #  2.  The 
symbology  will  include  all  navigation  symbology,  weapons  delivery  symbology,  a  manual 
terrain  following  (TF)  box,  radar  altitude  scale,  and  Low  Altitude  Navigation  and  Targeting 
Infra-Red  for  Night  (LANTIRN)  navigation  pod  video.  The  dimensions  of  the  HUD  FOV  were 
21°  in  elevation  and  28°  in  azimuth.  An  example  of  HUD  navigation  symbology  is  shown  in 
Figure  1.3.1. 


REMARKS/LESSONS  LEARNED 


The  above  symbology  was  chosen  for  both  the  baseline  and  the  IMPACT  cockpit  simulations. 
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HUD  AIR-TO-GROUND  SYMBOLOGY 


In  the  air-to-ground  master  mode,  the  manual  TF  box,  radar  altitude  scale,  and  bank  angle  scale 
are  removed  from  the  HUD.  The  continuously-displayed  impact  point  (CDIP)  pipper  is  added  as 
shown  in  Figure  1.3.2.  The  displayed  impact  line  (DIL)  indicates  the  path  where  the  bomb  will 
fall,  and  the  pipper  indicates  the  instantaneous  impact  point  for  the  bomb. 


Figure  1.3.2  Air-to-Ground  HUD  Symbology 


REMARKS/LESSONS  LEARNED 

The  above  symbology  was  chosen  for  both  the  baseline  and  the  IMPACT  cockpit  simulations. 
The  frag  cue  and  the  range  bar  shown  in  Figure  1.3.2  were  not  used  in  Advanced  Technology 
Experiment  #  2. 
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SPMR  1.4 


UP-FRONT  CONTROL  (UFC) 


INTRODUCTION 

The  UFC,  shown  in  Figure  1.4.1,  allows  entry  and  readout  of  critical  flight  information.  Located 
immediately  below  the  HUD,  it  consists  of  a  liquid  crystal  display  (LCD)  displaying  speed  and 
time,  a  keypad,  and  the  HUD  control  panel.  The  keypad  is  used  to  enter  data  into  the  up-front 
control  (UFC).  Data  are  displayed  in  the  scratchpad  area,  then  entered  into  the  system  when  the 
appropriate  pushbutton  (PB)  is  pressed.  The  UFC  pushbuttons  are  numbered  from  one  through 
ten,  starting  at  the  upper  left  comer  of  the  UFC  and  moving  counterclockwise  around  the  LCD. 

The  master  mode  buttons  for  air-to-air,  air-to-ground,  navigation  and  instrument  (A/A,  A A3, 
NAV,  and  INST  respectively)  are  “radio  button”  type  and  are  located  on  the  PIUD  control  panel 
below  the  UFC  keypad.  When  a  master  mode  button  is  pressed,  the  corresponding  pre¬ 
programmed  displays  appear  on  the  three  multi-purpose  displays  (MPD).  In  addition,  the 
appropriate  weapon  delivery  symbology  is  displayed  in  the  HUD.  The  INST  master  mode  will 
not  be  used  in  the  IMPACT  demonstrations. 
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STEERPOINT  LATITUDE,  LONGITUDE,  AND  ELEVATION  ENTRY 

To  enter  the  coordinates  and  elevation  of  a  new  steerpoint,  or  to  change  those  of  a  current 
steerpoint,  the  pilot  must  first  go  to  the  Point  Data  Lat/Long  submenu  as  shown  in  Figure  1.4.2. 
To  select  a  steerpoint  when  viewing  the  Data  1  display,  enter  the  steerpoint  number  in  the 
scratchpad,  then  press  PB  10  (or  PB  1  if  in  the  Lat/Long  submenu). 

To  enter  a  point’s  latitude  and  longitude,  enter  the  hemisphere,  degrees,  minutes,  and  tenths  of 
minutes.  When  the  hemisphere  key  (N,  S,  E,  or  W)  is  pressed,  the  degree  symbol,  minutes 
symbol,  decimal,  and  the  entered  letter  are  displayed  on  the  scratchpad.  Leading  zeroes  must  be 
included  if  the  latitude  degrees  are  less  than  two  digits  or  the  longitude  degrees  are  less  than 
three  digits.  Perform  the  following  actions  from  the  Data  1  display: 

•  Select  the  Lat/Long  submenu  Press  PB  10. 

•  Enter  the  new  latitude  For  example,  for  38  deg  45.6  north  latitude,  press 

SHF,  N,  38456,  then  press  latitude  button  (PB  2). 

•  Enter  the  new  longitude  For  example,  for  90  deg  22. 1  east  longitude,  press 

SHF,  E,  09022 1 ,  then  press  longitude  button  (PB  3). 

•  Enter  the  new  elevation  Type  the  new  elevation  in  the  scratchpad,  then  press 

the  button  next  to  the  steerpoint  elevation  (PB  7). 

•  Return  to  the  Data  1  display  Press  the  DATA  button  on  the  keypad. 


Figure  1 .4.2  Up-Front  Control,  Lat/Long  Submenu 

REMARKS/LESSONS  LEARNED 

The  UFC  was  implemented  as  a  virtual  display  on  the  simulator’s  large  color  CRT  for  Advanced 
Technology  Demonstration  #  2.  The  pushbuttons  were  not  functional,  and  the  display  was  not 
interactive.  Data  1  display  was  shown  continuously  with  a  combination  of  dynamic  and  static 
data. 
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SPMR  1.8 


TACTICAL  ELECTRONIC  WARFARE  SYSTEM  (TEWS) 
INTRODUCTION 

The  Tactical  Electronic  Warfare  System  (TEWS),  shown  in  Figure  1.8.1,  is  composed  of  a 
compass  rose  that  rotates  as  the  aircraft  turns,  displaying  current  aircraft  heading.  Threat 
warning  symbols  are  displayed  at  the  relative  bearing  and  approximate  range  from  the  aircraft  to 
the  threat  radar.  As  the  aircraft  turns,  the  threat  symbol  maintains  its  position  relative  to  the 
threat  radar.  The  INS  bearing  pointer  always  points  to  the  current  steerpoint,  and  the  bearing, 
range,  and  time  to  the  current  steerpoint  are  displayed  in  the  bottom  right  comer  of  the  display  in 
the  NAV  data  block. 


Figure  1.8.1  TEWS  display 


REMARKS/LESSONS  LEARNED 

The  TEWS  display  was  implemented  in  the  baseline  cockpit  as  described  above. 
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SPMR  1.9 


TERRAIN-FOLLOWING  (TF)  RADAR  DISPLAY 


INTRODUCTION 

The  terrain-following  (TF)  radar  display,  shown  in  Figure  1.9.1,  will  be  generated  based  on  the 
DTED  database.  A  manual  TF  box  will  be  displayed  on  this  display  and  in  the  HUD  for  the  pilot 
to  maintain  terrain  clearance.  The  radar  altitude  and  the  aircraft  groundspeed  are  displayed.  The 
pilot  can  choose  a  Set  Clearance  Limit  (SCL)  by  pressing  the  pushbutton  adjacent  to  the  desired 
radar  altitude.  This  altitude  is  then  boxed,  and  the  manual  TF  box  commands  flight  at  the 
selected  altitude. 


REMARKS/LESSONS  LEARNED 

The  TF  radar  display  was  implemented  in  both  the  baseline  (6”  square  format)  and  IMPACT 
cockpits  (5”  square  format)  as  described  above.  However,  because  the  display  was  simulated  on 
a  large-screen  CRT,  none  of  the  MPD  buttons  were  functional.  Therefore  the  pilot  was  unable  to 
change  any  of  the  display  functions,  including  the  set  clearance  limit.  Aa  a  result,  a  constant  300 
foot  set  clearance  limit,  with  smooth  ride,  was  selected  for  the  scenario  used  in  the  simulation 
and  the  obstacle  warning  line  was  not  implemented. 
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TACTICAL  SITUATION  DISPLAY  (TSD) 


INTRODUCTION 

The  Tactical  Situation  Display  (TSD),  shown  in  Figure  1.10.1,  is  composed  of  the  mission  route 
and  the  steerpoint  symbols  displayed  over  a  moving  map  (1:500,000  Tactical  Pilotage  Chart). 
The  display  ranges  are  10,  20,  40,  and  80  NM,  and  are  selected  by  pressing  PB  13  and  14.  The 
steerpoint  symbols  are  circles,  and  the  target  symbol  is  a  triangle.  Each  steerpoint  symbol  is 
labeled,  and  the  heading  to  the  next  steerpoint  is  displayed  at  the  beginning  of  each  segment. 
The  aircraft  symbol  is  located  1/4  of  the  display  height  from  the  bottom  of  the  display  when  BOT 
is  displayed  at  PB  6.  By  pressing  PB  6,  the  aircraft  symbol  is  placed  in  the  center  of  the  MPD 
and  CTR  is  displayed.  The  pilot  can  choose  a  bank  angle  and  airspeed  to  fly  the  mission,  and  the 
TSD  will  draw  the  route  with  the  turn  radii  for  that  bank  angle  and  airspeed. 


The  TSD  was  implemented  as  described  above,  with  the  following  exceptions:  Because  the 
display  was  simulated  on  a  large-screen  CRT,  none  of  the  MPD  buttons  were  functional. 
Therefore  the  pilot  was  unable  to  change  any  of  the  display  functions.  A  fixed  20  NM  range 
display  was  used,  and  the  aircraft  symbol  was  centered.  The  moving  map  presentation  was  not 
available,  so  a  plain  black  background  was  used  for  the  route  depiction. 
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ATTITUDE  DIRECTOR  INDICATOR  (ADI) 


INTRODUCTION 

The  attitude  director  indicator  (ADI)  display  contains  the  ADI,  a  heading  scale,  a  terrain 
following  pitch  steering  bar,  and  a  radar  altitude  scale  (see  Figure  1.12.1).  These  elements  are 
generated  by  the  aero  model,  and  move  in  real-time  as  the  pilot  flies  the  aircraft.  Calibrated 
airspeed  is  boxed  and  displayed  on  the  left  side  of  the  ADI.  The  barometric  altitude  is  boxed  and 
displayed  on  the  right  side  of  the  ADI,  and  the  radar  altitude  is  displayed  below  it,  preceded  by 

an  “R  ” 


REMARKS/LESSONS  LEARNED 

The  ADI  display  was  presented  (on  the  simulator’s  large  screen  CRT)  in  the  position  of  the  left- 
hand  monochrome  MPD.  Concerns  about  the  way  the  ADI  ball  was  depicted  were  expressed  by 
test  subjects  and  members  of  the  experiment  team.  In  the  monochrome  version,  the  bottom  of 
the  ball  was  drawn  with  a  series  of  closely  spaced  parallel  (green)  lines.  In  an  unusual  attitude, 
this  was  slightly  misleading  since  the  bright  half  of  the  ADI,  which  is  normally  expected  to 
indicate  the  sky,  indicated  the  ground. 
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SPMR  1.14 


ENGINE  MONITOR  DISPLAY 


INTRODUCTION 

The  engine  monitor  display,  shown  in  Figure  1.14.1,  contains  the  engine  RPM,  temperature,  fuel 
flow,  nozzle  position,  and  oil  pressure  displayed  on  a  single  Liquid  Crystal  Display  (LCD). 


Figure  1 . 14. 1  Engine  Monitor  Display 
REMARKS/LESSONS  LEARNED 
The  display  was  implemented  as  described  above. 
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SPMR2.0 


COCKPIT  LAYOUT 


INTRODUCTION 

The  IMPACT  cockpit  is  characterized  by  two  10  x  10  inch  color  displays.  Each  display  can  be 
divided  into  four  5x5  inch  displays.  Each  10-inch  display  has  40  pushbuttons  (PBs),  ten  on 
each  side.  They  are  numbered  from  1  to  40,  starting  at  the  upper  left  comer  of  the  display  and 
moving  counterclockwise  around  the  display,  as  shown  in  Figure  2.0.1.  Possible  configurations 
are  shown  in  Figures  2.0.2  and  2.0.3. 


Figure  2.0.1  Pushbutton  Numbering  Scheme 


Figure  2.0.2  Possible  10x10  Inch  Display  Configurations 
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Figure  2.0.3  Possible  IMPACT  Cockpit  Display  Configuration 
REMARKS/LESSONS  LEARNED 

A  single  display  configuration  was  chosen  for  the  IMPACT  cockpit,  tailored  to  the  low-level 
penetration,  threat  evasion,  and  weapon  delivery  tasks  associated  with  the  experiment.  The 
configuration  included  all  the  elements  shown  in  Figure  2.0.3,  except  for  the  air-to-air  display  in 
the  upper  left  comer.  It  was  replaced  by  a  terrain-following  radar  display,  described  in  SPMR 
2.9.  The  pushbuttons  were  not  functional  because  the  displays  were  simulated  on  a  large  screen 
CRT,  and  pilots  were  not  required  to  change  any  of  the  displays. 
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SPMR  2.1 


IMPACT  STICK  AND  THROTTLES 


STICK 

An  F-15E  stick  grip  will  be  used  in  the  IMPACT  cockpit  for  Advanced  Technology  Experiment 
#  2.  The  stick  will  be  fully  operational  for  control  of  aircraft  pitch  and  roll.  The  switches  on  the 
stick  grip  that  must  function  for  the  first  demonstration  are  the  trim  switch  and  the  castle  switch. 
The  trim  switch  is  used  for  aircraft  trim  and  countermeasures  dispense,  and  the  castle  switch  is 
used  for  pre-programmed  display  rotation.  (See  Figure  2.1.1). 


REMARKS/LESSONS  LEARNED 

The  grip  for  the  side-stick  used  in  the  actual  simulation  was  slightly  different  from  the 
illustration.  Because  the  pilot  was  not  required  to  change  the  display  formats  for  Advanced 
Technology  Experiment  #  2,  the  castle  switch  function  was  not  implemented.  After  pre- 
experimental  sessions  revealed  minor  problems  with  planned  methods  for  countermeasures 
dispense  function,  the  design  team  decided  to  use  a  paddle  switch  (not  illustrated),  which  was 
located  at  the  front  base  of  the  stick  to  execute  the  function. 
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THROTTLES 


The  throttles  used  in  the  IMPACT  cockpit  are  based  on  the  F-15E  throttles.  They  must  be 
capable  of  controlling  engine  rpm,  operating  speedbrakes,  and  dispensing  countermeasures. 

The  speedbrake  switch  has  three  positions:  FORWARD,  CENTER,  and  AFT. 

•  FORWARD  Retracts  speedbrakes 

•  CENTER  Holds  speedbrakes  in  current  position 

•  AFT  Extends  speedbrakes 

The  CMD  switch  has  two  positions:  UP  and  DOWN 

•  UP  Manual  2  dispense 

•  DOWN  Manual  1  dispense 


REMARKS/LESSONS  LEARNED 

The  throttles  in  the  simulator  used  for  Advanced  Technology  Experiment  #  2  were  slightly 
different  in  configuration  from  the  F-l  5E,  but  had  functionally  equivalent  switches.  It  was 
originally  planned  to  use  the  “Pinky”  switch  to  dispense  chaff  in  the  experiment.  However,  due 
to  the  awkward  motion  required  to  operate  this  switch  and  the  need  to  obtain  precise  data 
concerning  the  timing  of  chaff  dispensing,  the  team  decided  to  use  the  paddle  switch  on  the 
control  stick  instead. 


A-21 


SPMR  2.3 


HEAD-UP  DISPLAY  (HUD) 


INTRODUCTION 

F-15E  HUD  symbology  will  be  used  for  IMPACT  Advanced  Technology  Experiment  #  2.  The 
symbology  will  include  all  navigation  symbology,  weapons  delivery  symbology,  a  manual 
terrain  following  (TF)  box,  radar  altitude  scale,  and  Low  Altitude  Navigation  and  Targeting 
Infra-Red  for  Night  (LANTIRN)  navigation  pod  video.  The  dimensions  of  the  HUD  FOV  were 
21°  in  elevation  and  28°  in  azimuth.  An  example  of  HUD  navigation  symbology  is  shown  in 
Figure  2.3.1. 


REMARKS/LESSONS  LEARNED 

The  above  symbology  was  chosen  for  both  the  baseline  and  the  IMP  ACT  cockpit  simulations. 
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HUD  AIR-TO-GROUND  SYMBOLOGY 

In  the  air-to-ground  master  mode,  the  manual  TF  box,  radar  altitude  scale,  and  bank  angle  scale 
are  removed  from  the  HUD.  The  continuously-displayed  impact  point  (CDEP)  pipper  is  added  as 
shown  in  Figure  2.3.2.  The  displayed  impact  line  (DIL)  indicates  the  path  where  the  bomb  will 
fall,  and  the  pipper  indicates  the  instantaneous  impact  point  for  the  bomb. 


Figure  2.3.2  Air-to-Ground  HUD  Symbology 


REMARKS/LESSONS  LEARNED 

The  above  symbology  was  chosen  for  both  the  baseline  and  the  IMPACT  cockpit  simulations. 
The  frag  cue  and  the  range  bar  shown  in  Figure  2.3.2  were  not  used  in  Advanced  Technology 
Experiment  #  2. 
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SPMR  2.4 


UP-FRONT  CONTROL  (UFC) 


INTRODUCTION 

The  display  surface  for  the  Up-Front  Control  (UFC)  in  the  IMPACT  cockpit  will  be  identical  in 
form  and  function  to  the  baseline  UFC.  (See  SPMR  1.4.)  It  is  illustrated  in  Figure  2.4.1. 
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Figure  2.4.1  IMPACT  UFC  Data  1  Display 


REMARKS/LESSONS  LEARNED 

The  UFC  for  the  IMPACT  cockpit  was  an  expedient  design  used  to  provide  dynamic  textual 
information  comparable  to  that  available  in  the  baseline  cockpit.  No  keypad  design  was 
incorporated,  since  it  was  known  at  the  time  the  UFC  was  designed  so  that  the  pilot  would  not  be 
required  to  interact  with  the  display.  Sufficient  space  exists  to  include  a  keypad  in  the  vicinity  of 
the  display  surface.  The  UFC  is  also  a  possible  candidate  for  implementation  of  voice  data  entry. 
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SPMR  2.8 


THREE-DIMENSIONAL  (3-D)  AUDIO 


INTRODUCTION 

The  three-dimensional  (3-D)  audio  capability  for  the  IMPACT  simulation  shall  be  implemented 
via  the  Auditory  Localization  Cue  Synthesizer  (ALCS)  system  designed  by  Armstrong 
Laboratory.  The  ALCS  encodes  naturally  occurring  spatial  information  into  an  audio  signal  and 
presents  a  sound  source  that  can  vary  in  azimuth  over  stereo  headphones  incorporated  into  the 
pilot’s  helmet.  The  encoded  signal  sounds  to  the  pilot  as  if  it  originated  from  a  particular 
location.  As  the  pilot  changes  head  orientation,  the  relative  position  of  the  sound  source  is 
updated  to  make  the  sound  appear  stationary  with  the  listening  environment. 

At  least  two  different  sounds  will  be  used  in  the  3-D  audio  system:  one  for  the  surface-to-air 
threat  and  one  for  the  air-to-air  threat.  The  sounds  will  be  activated  simultaneously  with  the 
heads-down  threat  warning  display  of  enemy  radar  lock-on  and  will  stop  on  the  test  operator’s 
input. 

REMARKS/LESSONS  LEARNED 

Several  changes  were  made  in  the  implementation  of  the  3-D  audio  system  as  a  result  of  pre- 
experimental  study.  The  most  significant  change  was  the  elimination  of  the  elevation  cue, 
resulting  in  a  strictly  azimuthal  audio  system.  The  change  was  made  because  of  the  necessity  to 
operate  the  simulator  intercom  system  through  the  same  audio  processing  equipment  as  the 
directional  audio  system.  Additionally,  the  sounds  used  in  the  system  were  changed  significantly 
after  consultation  with  representatives  from  Armstrong  Laboratory.  In  an  effort  to  provide  better 
directional  resolution,  the  clear  tones  initially  chosen  for  threat  annunciation  were  replaced  with 
multi-spectral  “pink”  noise.  The  same  intermittent  sound  was  used  for  both  surface  and  air 
threats,  since  the  localization  task  was  essentially  identical  for  both  types  of  threats.  The  sound 
continued  as  long  as  a  threat  was  active.  Armstrong  Laboratory  also  provided  new  headphones 
to  replace  the  relatively  inexpensive  ones  supplied  with  the  helmet.  The  new  headphones 
provided  significantly  better  sound  quality  and  helmet  fit. 
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SPMR  2.9 


TERRAIN-FOLLOWING  (TF)  RADAR 


INTRODUCTION 

The  IMPACT  Terrain-Following  (TF)  radar  display,  shown  in  Figure  2.9.1,  will  be  generated 
based  on  the  DTED  database.  A  manual  TF  command  box  will  be  displayed  here  and  in  the 
HUD  to  aid  the  pilot  in  maintaining  terrain  clearance. 


DISPLAY  MENU  SELECT 

To  select  the  Display  Menu,  the  pilot  will  press  (or  use  the  TDC  to  click)  the  “M”  pushbutton  at 
the  lower  right  comer  of  the  display. 

REMARKS/LESSONS  LEARNED 

The  TF  radar  display  was  implemented  as  described  above,  with  the  following  exceptions:  The 
pilot  was  neither  required  nor  able  to  change  any  of  the  display  functions,  including  the  set 
clearance  limit.  A  constant  300  foot  set  clearance  limit,  with  smooth  ride,  was  selected  for  the 
scenario  used  in  the  simulation.  The  obstacle  warning  line  was  not  implemented. 

Note:  The  baseline  and  IMPACT  TFR  displays  were  essentially  the  same  except  that  the 
baseline  was  a  6”  square  format  and  the  IMPACT  was  a  5”  square  format. 
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SPMR2.10 


TACTICAL  SITUATION  DISPLAY  (TSD) 


INTRODUCTION 

The  tactical  situation  display  (TSD),  shown  in  Figure  2.10.1,  consists  of  the  mission  route  and 
steerpoint  symbols  displayed  over  a  computer-generated  moving  map.  The  terrain  features  will 
come  from  the  DTED  database.  Threat  rings  are  positioned  at  the  known  threat  locations.  The 
display  ranges  are  1 0,  20,  40,  80,  and  1 60  NM. 

The  steerpoint  symbols  are  circles,  and  the  target  symbol  is  a  triangle.  Each  steerpoint  symbol  is 
labeled,  and  the  heading  to  the  next  steerpoint  is  displayed  at  the  beginning  of  each  segment. 
The  aircraft  symbol  is  centered  in  a  compass  rose  ana  is  located  1/4  of  the  display  height  from 
the  bottom.  The  pilot  shall  have  the  ability  to  place  the  symbol  in  the  center  of  the  display,  if 
desired.  Friendly  and  enemy  aircraft  are  displayed  on  the  TSD  based  on  ownship  radar  or  data- 
linked  information. 

When  the  aircraft  is  retasked  inflight,  a  message  will  appear  on  the  TSD  to  alert  the  pilot.  By 
selecting  pushbuttons,  the  pilot  can  preview  the  new  route,  time,  fuel,  weather,  threats,  and 
support  asset  locations.  Specifics  for  the  mechanization  of  these  functions  will  be  included  in 
updates  to  this  SPMR. 


Figure  2.10.1  Tactical  Situation  Display 


SURFACE-TO-AIR  THREATS 

Surface-to-air  threats  will  be  displayed  on  the  TSD  based  on  pre-mission  planning,  data-linked 
information,  or  ownship  estimate  of  location  after  a  threat  radar  locks  onto  the  aircraft.  These 
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threats  will  be  displayed  as  hexagons,  with  the  identifier  in  the  center  (Figure  2.10.2).  After  a 
threat  locks  onto  the  aircraft,  a  solid  RED  line  will  connect  the  threat  symbol  and  the  aircraft 
symbol  (Figure  2.10.3).  If  there  is  ambiguity  as  to  whom  the  threat  has  locked,  two  solid  lines 
will  “fan  out”  from  the  threat  symbol  (Figure  2.10.4). 


Figure  2.10.2  Surface-to-Air  Threat  Symbol 


Figure  2.10.3  Threat  Locked  Onto  Aircraft  Figure  2. 1 0.4  Threat  Lock-On  Ambiguity 


The  assumption  is  that  advanced  fighters  will  have  on-board  systems  that  can  detect  and  track 
enemy  surface-to-air  missiles  inflight.  On  the  IMPACT  TSD,  a  surface-to-air  missile  (SAM) 
will  be  displayed  as  a  solid  RED  triangle  with  a  velocity  vector,  connected  to  the  threat  symbol 
by  an  ORANGE  line.  The  missile's  relative  elevation  (in  degrees)  will  be  displayed  above  the 
threat  symbol,  and  time  to  impact  (in  seconds)  will  be  displayed  below  it  (Figure  2.10.5). 


Figure  2.10.5  Missile  Elevation  (+15  deg)  and  Time  to  Impact  (9  sec) 

If  the  aircraft  maneuvers  after  missile  launch,  the  lethality  of  the  threat  decreases  (even  though 
the  threat  radar  may  still  be  locked  onto  the  aircraft).  This  decrease  in  Pk  is  shown  by  dashed, 
then  dotted  lines  from  the  threat  symbol  to  the  aircraft  symbol  (Figure  2.10.6).  If  aircraft 
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maneuvers,  ECM  or  expendables  break  the  lock  of  the  radar  or  missile,  the  line  is  removed  from 
the  TSD.  The  missile  will  be  displayed  as  long  as  the  aircraft  is  able  to  track  it. 


The  TSD  will  automatically  change  to  1 0  NM  range  scale  when  a  missile  inflight  reaches  1 0  NM 
from  the  aircraft  (Figure  2.10.7).  If  another  threat  locks  onto  the  aircraft,  the  pilot  must  expand 
the  TSD  scale  to  determine  the  threat  type  and  location. 


DISPLAY  OF  AIRCRAFT 


Aircraft  will  be  displayed  on  the  TSD  based  on  ownship  radar  information,  data-linked 
information,  or  ownship  estimate  of  location  after  an  airborne  radar  locks  onto  the  aircraft. 
Friendly  aircraft  have  a  unique  symbol  (GREEN  circle  with  velocity  vector).  Unknown  aircraft 
will  be  ORANGE,  and  enemy  aircraft  will  be  RED.  Their  airspeed  will  be  shown  using  distinct 
symbols  (Figure  2.10.8).  The  symbols  will  include  an  aircraft  identifier  (if  known)  and  altitude. 
The  heading  of  the  airborne  contact  is  denoted  by  the  direction  in  which  the  symbol  is  pointing. 
Future  updates  to  this  SPMR  will  include  symbology  for  air-to-air  missiles  inflight. 
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(MiG  29  at  29,600  feet, 

color  ORANGE) 

color  RED) 

Figure  2.10.8  TSD  Aircraft  Symbols 


REMARKS/LESSONS  LEARNED 

In  Advanced  Technology  Experiment  #  2,  a  simplified  version  of  the  TSD  was  implemented.  In 
the  scenario  used  during  the  experiment,  the  airborne  and  SAM  threats  were  stationary  in  space, 
so  the  symbols  used  to  represent  them  moved  only  as  a  result  of  aircraft  motion.  Missile  launch 
symbology  like  that  shown  in  Figure  2.10.5  was  used  to  represent  surface-to-air  threats,  and 
“fast”  aircraft  symbols  (like  the  MiG-29  in  Figure  2.10.8)  represented  the  air-to-air  threats.  The 
TSD  range  was  fixed  at  40  NM. 
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SPMR2.il 


WEAPONS/COUNTERMEASURES  (WPNS/CM)  DISPLAY 
INTRODUCTION 

The  Weapons/Countermeasures  (WPNS/CM)  display,  shown  in  Figure  2.11.1,  contains  the 
following  data:  a  graphical  representation  of  all  weapons  loaded  on  the  aircraft;  a  RDY  (ready) 
indication  when  all  pre-release  functions  have  been  met;  chaff  and  flare  counters  with  the 
bundles  remaining  highlighted;  the  release  pulse  selected  (single,  multi,  or  program);  and  the 
internal  electronic  countermeasures  (ECM)  status  (off,  standby,  operate). 

The  WPNS/CM  display  will  be  a  static  picture  for  Advanced  Technology  Experiment  #  2. 


Figure  2.11.1  WPNS/CM  Display 


DISPLAY  MENU  SELECT 

To  select  the  Display  Menu,  press  (or  use  the  TDC  to  click)  the  “M”  pushbutton  at  the  lower 
right  comer  of  the  display. 

REMARKS/LESSONS  LEARNED 

The  WPNS/CM  display  used  in  Experiment  #  2  appeared  as  shown  in  Figure  2.11.1.  Two 
functions  on  the  display  were  dynamic  to  provide  feedback  to  the  pilot:  1)  The  chaff  display 
indicated  expenditure  of  one  bundle  each  time  the  paddle  switch  (chaff  dispense)  was  depressed, 
and  2)  The  Mk-84  icons  were  removed  when  the  pilot  simulated  release  of  the  weapons. 
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SPMR  2.12 


ATTITUDE  DIRECTOR  INDICATOR  (ADI) 

INTRODUCTION 

The  attitude  director  indicator  (ADI)  display  contains  the  ADI,  a  heading  scale,  a  terrain¬ 
following  pitch  steering  bar,  and  a  radar  altitude  scale,  as  shown  in  Figure  2.12.1.  These  objects 
are  generated  by  the  aero  model  and  move  in  real-time  as  the  pilot  flies  the  aircraft.  Calibrated 
airspeed  is  boxed  and  displayed  on  the  left  side  of  the  ADI.  The  barometric  altitude  is  boxed  and 
displayed  on  the  right  side  of  the  ADI,  and  the  radar  altitude  is  displayed  below  it,  preceded  by 
an  “R.” 


Figure  2.12.1  ADI 


DISPLAY  MENU  SELECT 

To  select  the  Display  Menu,  press  (or  use  the  TDC  to  click)  the  “M”  pushbutton  at  the  lower 
right  comer  of  the  display. 

REMARKS/LESSONS  LEARNED 

The  ADI  was  implemented  as  described  above  for  Advanced  Technology  Experiment  #  2.  The 
color  version  of  this  ADI  (based  on  the  F-15E  color  scheme)  was  placed  in  the  upper  right  comer 
of  the  left  10  x  10  inch  display  surface  on  the  simulator’s  large-screen  CRT  (simulated 
instmment  panel).  The  color  version  of  this  ADI  was  easier  to  interpret  in  unusual  attitudes  than 
the  version  placed  on  the  monochrome  display  as  described  in  SPMR  1.12. 
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SPMR  2.14 


ENGINE/FUEL  DISPLAY 

INTRODUCTION 

The  engine/fuel  display,  shown  in  Figure  2.14.1,  contains  graphic  representations  of  the  two 

engines  and  the  fuel  tanks.  The  following  parameters  are  shown: 

•  Engine  speed  in  per  cent  RPM,  fuel  flow  (FF)  in  pounds  per  hour,  temperature  (TEMP)  in 
degrees  Celsius,  nozzle  position  (NOZ)  in  per  cent  open,  oil  pressure  (OIL)  in  pounds  per 
square  inch,  and  hydraulic  pressure  (HYD)  in  pounds  per  square  inch  for  the  aircraft’s  two 
hydraulic  systems. 

•  Total  fuel  quantity  in  pounds,  displayed  above  the  fuel  tank  graphic,  and  a  pictorial 
representation  of  the  amount  of  fuel  remaining  in  each  tank  (internal  and  external  wing  tanks, 
internal  fuselage  tanks). 


F igure  2.14.1  Engine/F uel  Display 


FUEL  QUANTITY  SELECT 

The  pilot  can  determine  the  amount  of  fuel  remaining  in  the  internal  and  external  tanks  by 
selecting  the  INT  WING,  EXT  WING,  or  FUSELAGE  pushbuttons.  When  selected,  those  tanks 
will  be  highlighted,  and  the  amount  of  fuel  remaining  will  be  displayed  at  the  top  of  the  fuel 
display.  To  return  to  the  total  fuel  quantity  display,  deselect  the  pushbutton. 

DISPLAY  MENU  SELECT 

To  select  the  Display  Menu,  press  (or  use  the  TDC  to  click)  the  “M”  pushbutton  at  the  lower 
right  comer  of  the  display. 
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REMARKS/LESSONS  LEARNED 


This  display’s  appearance  was  implemented  as  described  above  for  Advanced  Technology 
Experiment  #  2.  The  pilot  was  not  required  to  interact  with  the  display  by  changing  display 
modes  or  selecting  the  Display  Menu,  so  those  capabilities  were  not  implemented. 
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SPMR  2.15 


HELMET-MOUNTED  DISPLAY  (HMD) 


INTRODUCTION 

The  Kaiser  SIM-EYE  40  helmet-mounted  display  (HMD)  will  be  used  to  display  navigation, 
threat  warning,  air-to-ground  weapon  delivery,  and  general  situation  awareness  symbology.  The 
HMD  will  also  be  used  to  display  threat  symbols  for  SAMs  and  air  threats  in  both  the  IMPACT 
and  baseline  cockpits  (see  SPMR  3.5).  This  SPMR  focuses  on  the  symbology  requirements  for 
the  HMD. 

HMD  SYMBOLOGY  MAPPING  ORIENTATION 

Figure  2.15.1  illustrates  the  reference  grid  for  mapping  and  placement  of  the  HMD  symbols  and 
numeric  information.  The  total  dimensions  are  based  on  two  40°  circular  FOV  displays,  each 
with  a  resolution  of  1280  x  1024  and  a  50%  overlap  in  the  horizontal  dimension.  The  grid’s 
dimensions  are  milliradians  (mr).  It  measures  1 047  mr  horizontally  by  698  mr  vertically.  All 
symbology  locations  in  this  SPMR  are  referenced  to  the  origin  of  the  grid  (coordinates  0,0). 

698  milliradians 


F  i  gure  2.15.1  HMD  Grid  Reference 

Figure  2.15.2  illustrates  the  NAV  master  mode  symbology  requirements.  The  NAV  master 
mode  display  includes  dedicated  windows  for  digital  information  and  dynamic  symbology  as  the 
situation  warrants.  When  the  pilot  is  looking  within  ±  108  degrees  of  the  HUD  boresight  the 
HMD  symbology  should  be  removed.  The  following  paragraphs  describe  each  requirement  in 
detail. 
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DEDICATED  WINDOWS 

Master  Arm  Indication  -  The  legend  “ARM”  shall  appear  in  a  dedicated  window  whenever  the 
master  arm  switch  is  in  the  ARM  position.  The  lower  left  comer  of  the  dedicated  window  should 
be  located  at  approximately  -430,-212  mr  from  the  center  point.  The  character  height  for  the 
legend  “ARM”  shall  be  7  mr  (24  minutes  of  arc). 

Mach  Indication  -  The  mach  indication  shall  appear  at  all  times  and  is  designated  with  an  “M 
followed  by  the  current  mach  value  (to  the  second  decimal  place).  The  lower  left  comer  of  the 
dedicated  window  should  be  located  approximately  -430,-221  mr  from  the  center  point.  The 
character  height  for  the  mach  information  shall  be  7  mr  (24  minutes  of  arc). 

Airspeed  Indication  -  The  airspeed  indication  shall  be  displayed  at  all  times  as  a  digital 
indication  of  current  calibrated  airspeed.  The  lower  left  comer  of  the  dedicated  window  should 
be  located  at  approximately  -430,-230  mr  from  the  center  point.  The  character  height  for  the 
airspeed  information  shall  be  7  mr  (24  minutes  of  arc). 

Altitude  Indication  -  The  digital  altitude  indication  shall  be  displayed  at  all  times.  The  digital 
attitude  indication  shall  indicate  radar  altitude  any  time  the  aircraft  is  below  5000’  AGL  and  shall 
be  denoted  be  the  legend  “R”  immediately  preceding  the  digital  indication.  The  digital 
indication  shall  display  baro  altitude  any  time  the  aircraft  is  above  5000’  AGL.  The  digital 
information  shall  appear  in  a  dedicated  window  that  has  the  lower  left  comer  positioned  at 
approximately  340,-230  mr  from  the  center  point.  The  character  height  for  the  altitude 
information  shall  be  7  mr  (24  minutes  of  arc). 

HMD  SYMBOLOGY 

Attitude  Indication  -  The  orange  peel  (Figure  2.15.3)  shall  be  displayed  at  all  times  and  is  used 
to  indicate  the  aircraft  attitude.  The  symbol  consists  of  a  “winged”  arc  and  an  aircraft  symbol 
centered  at  0,-210. 
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Figure  2. 1 5.3  Orange  Peel 

Figure  2.15.4  illustrates  the  dynamics  of  the  orange  peel  during  a  loop.  At  the  start  point,  the 
wings  of  the  arc  are  in  line  with  the  wings  of  the  aircraft  symbol.  As  the  loop  is  initiated,  the 
angle  subtended  by  the  arc  decreases  such  that  at  the  zenith  the  arc  nearly  disappears,  leaving 
only  the  aircraft  symbol  wings  visible.  As  the  aircraft  continues  through  the  loop,  the  arc  flips  to 
the  other  side  of  the  aircraft  symbol.  During  the  back  portion  of  the  loop  the  angle  subtended  by 
the  arc  increases.  When  at  the  horizon  and  when  inverted,  the  arc  is  positioned  above  the  aircraft 
symbol  with  its  wings  in  line  with  the  wings  of  the  aircraft  symbol.  As  the  nose  passes  through 
the  horizon,  the  angle  subtended  by  the  arc  continues  to  increase  until,  when  pointed  straight 
down,  the  aircraft  symbol  is  completely  enclosed  by  the  arc. 
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Figure  2.15.4  Orange  Peel  During  A  Loop 


Roll  is  indicated  by  the  opening  of  the  arc  rotating  around  the  aircraft  symbol,  as  indicated  in 
Figure  2.15.5. 
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Figure  2.15.5  Orange  Peel  Indicating  Roll 
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Altitude  Thermometer  -  The  radar  altitude  thermometer  (Figure  2.15.6)  shall  be  displayed 
whenever  the  radar  altimeter  is  on  and  the  aircraft  is  at  1500’  or  below.  The  character  height  for 
the  digital  information  should  be  4  mr  (16  minutes  of  arc).  The  thermometer  tape  shall 
dynamically  grow  or  shrink  to  reflect  the  current  radar  altitude.  The  thermometer  shall  be 
positioned  directly  above  the  digital  altitude  indication  (Approximately  3  mr  separation  between 
the  0  tick  mark  and  the  digital  indication). 


Figure  2. 1 5.6  Radar  Altitude  Thermometer 

Aiming  Cross  -  The  aiming  cross  (Figure  2.15.7)  indicates  the  boresight  of  the  HMD  and  shall 
always  be  displayed.  The  aiming  cross  shall  be  centered  at  0,0  and  will  be  used  to  designate 
points  of  interest. 
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Figure  2.15.7  Aiming  Cross 


Horizon  Line  -  The  horizon  line  (Figure  2.15.9)  shall  be  displayed  at  all  times  and  shall  be 
aligned  with  the  true  horizon.  When  the  horizon  is  outside  the  HMD  field-of-view  (FOV)  the 
horizon  line  shall  become  dashed  and  caged  at  the  outer  limits  of  the  HMD. 


Figure  2.15.9  Horizon  line 
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Steerpoint  Indicator  -  The  steerpoint  indicator  (Figure  2.15.10)  is  used  for  navigation  to  aid  the 
pilot  in  visually  acquiring  waypoints.  The  symbol  is  dynamic  and  represents  the  position  of  the 
next  steer-to-point.  When  the  pilot’s  head  is  positioned  so  that  the  steerpoint  is  no  longer  within 
the  HMD  FOV,  the  steerpoint  indicator  shall  be  caged  to  the  perimeter  of  the  HMD. 


12mr 


Figure  2.15.10  Steerpoint  Indicator 


Threat  Warning  Indicator  -  The  threat  warning  indicator  (Figure  2.15.1 1)  shall  be  displayed 
whenever  the  radar  warning  receiver  indicates  that  a  threat  has  been  detected.  It  shall  be 
dynamic  and  represent  the  location  of  the  threat. 

8mr 

JD~ 
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Figure  2.15.1 1  Threat  Warning  Indicator 

Threat  Locator  Indicator  -  The  threat  locator  indicator  (Figure  2.15.12)  shall  be  displayed 
whenever  the  threat  is  outside  the  HMD  FOV.  The  tip  of  the  lower  chevron  shall  be  displayed 
10  mr  inside  the  HMD  perimeter  and  in  the  direction  of  the  threat.  When  the  threat  is  located 
behind  the  aircraft  the  chevron  symbol  shall  become  dashed. 
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Figure  2.15.12  Threat  Locator  Indicator 

A/G  Weapon  Delivery  Symbology  -  When  the  air-to-ground  (A/G)  master  mode  is  selected  the 
radar  altitude  thermometer  shall  be  removed  and  the  following  symbology  shall  be  displayed  as 
follows: 

a.  Target  Designator  Diamond  -  The  target  designator  diamond  (Figure  2.15.13)  shall  be 
displayed  when  the  pilot  selects  A/G  master  mode  and  shall  dynamically  indicate  the  target 
location. 
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Figure  2.15.13  Target  Designator  Diamond 

b.  Target  Locator  Line  -  The  target  locator  line  (Figure  2.15.14)  shall  appear  whenever  the 
target  is  outside  the  HMD  FOV.  It  shall  originate  from  the  center  point  (0,0)  of  the  HMD 
and  point  in  the  direction  of  the  designated  target.  The  target  designator  diamond  shall  be 
removed  when  the  target  locator  line  is  displayed. 


c.  Target  Range  Indication  -  The  target  range  indication  shall  appear  when  the  A/G  master 
mode  is  selected.  The  digital  target  range  information  shall  appear  directly  under  the  target 
designator  diamond  or  the  tip  of  the  target  locator  line.  The  character  height  for  the  target 
range  information  shall  be  7  mr  (24  minutes  of  arc). 

REMARKS/LESSONS  LEARNED 

Several  changes  were  made  to  the  HMD  symbology  as  a  result  of  pre-experimental  evaluation  of 

the  display.  The  resulting  display  had  the  arrangement  shown  in  Figure  2.15.15.  The  following 

changes  were  implemented: 

•  A  bi-ocular  display  was  used  instead  of  the  planned  50%  overlapped  binocular  display.  This 
was  necessary  because  of  limitations  on  the  number  of  computer  video  channels  available. 
The  same  40°  circular  FOV  display  was  presented  to  both  of  the  pilot’s  eyes.  Fixed 
symbology  remained  at  approximately  the  same  distance  from  each  edge  of  the  display  as  in 
the  binocular  version.  The  orange  peel  was  located  immediately  below  the  center  of  the 
display. 

•  The  orange  peel  was  doubled  in  size.  The  original  symbol  was  determined  too  small  to  be 
useful. 

•  The  TF  thermometer  was  moved  50  mr  higher  in  the  display  to  align  it  more  naturally  with 
the  pilot’s  normal  line  of  sight. 

•  The  threat  locator  indicator  (Figure  2.15.12)  was  replaced  with  a  threat  locator  line  that 
extended  from  the  center  of  the  HMD  FOV  in  the  direction  of  the  threat.  The  threat  locator 
line  remained  visible  until  the  threat  warning  symbol  was  within  the  central  20°  (+\-10°  of 
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center)  of  the  HMD  FOV.  This  change  was  made  because  the  threat  locator  indicator,  at  the 
perimeter  of  the  HMD  FOV,  was  difficult  to  see  whenever  the  HMD’s  exit  pupil  was 
displaced  by  minor  helmet  slippage.  The  simultaneous  presentation  of  the  locator  line  and 
the  threat  warning  symbol  in  the  outer  10°  FOV  band  eliminated  the  possibility  that  the  line 
would  disappear  before  the  pilot  could  see  the  symbol. 


A-42 


SPMR  3.0 


HUD  DISPLAY  ON  BARCO  PROJECTION  SCREEN 
INTRODUCTION 

The  HUD  symbology  described  in  SPMRs  1 .3  and  2.3  will  be  presented  on  a  Barco  projection 
screen.  The  symbology  for  both  cockpits  is  referenced  to  the  screen  by  the  horizon  vision  line 
(see  Figure  3.0.1).  This  line  extends  horizontally  from  the  design  eye  of  the  simulator  to  the 
(laterally  measured)  center  of  the  projection  screen.  The  HUD  waterline  symbol  shall  be  located 
on  the  horizon  vision  line.  The  lower  limits  of  the  projection  shall  be  14.5°  below  the  horizon 
vision  line.  The  upper  limits  shall  be  6.5°  above  the  horizon  vision  line.  The  azimuth  limits 
shall  be  ±14°  from  the  horizon  vision  line. 

The  Barco  projection  screen  speaker  cover  shall  be  centered  and  located  12  inches  in  front  of  the 
simulator’s  nose. 


12" 


BARCO  SIMULATOR 

PROJECTION 

SCREEN 


Figure  3.0.1  Location  of  Barco  Projection  Screen  and  Simulator 


REMARKS/LESSONS  LEARNED 

The  arrangement,  shown  in  Figure  3.0.1,  was  satisfactorily  used  for  Advanced  Technology 
Experiment  #  2. 
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SPMR  3.1 


FORWARD-LOOKING  INFRA-RED  (FLIR)  DISPLAY 
INTRODUCTION 

A  simulated  Low  Altitude  Navigation  and  Targeting  Infra-Red  for  Night  (LANTIRN)  navigation 
pod  forward-looking  infra-red  (FLIR)  display  will  be  depicted  on  the  Barco  projection  screen, 
coincident  with  the  21  x  28  degree  HUD  presentation.  The  existing  visual  display  capability  of 
the  simulation  may  be  used.  It  should  be  presented  in  green  monochrome.  The  visibility  shall  be 
adjustable  by  the  software  programmers.  A  simulated  overcast  shall  restrict  the  view  of  the 
visual  scene  when  the  pilot  is  above  or  in  it. 


REMARKS/LESSONS  LEARNED 

The  arrangement  described  was  satisfactorily  employed  for  Advanced  Technology  Experiment 
#  2.  Some  adjustments  to  the  display  and  symbology  brightness  were  required  to  accommodate 
the  lack  of  background  contrast  when  the  simulated  aircraft  was  flying  through  the  clouds. 
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SPMR  3.2 


OUT-THE-WINDOW  DISPLAY 

INTRODUCTION 

Since  the  simulated  mission  is  at  night  there  will  not  be  an  out-the-window  scene.  The  only 
terrain  scene  visible  from  either  cockpit  will  be  the  HUD  FLIR  picture. 


REMARKS/LESSONS  LEARNED 

The  arrangement  described  was  satisfactorily  employed  for  Advanced  Technology  Experiment 

#2. 


A-45 


SPMR3.3 


FEATURE  ANALYSIS  DISPLAY 


INTRODUCTION 

Visual  scene  features  required  for  this  experiment  include  roads,  airfields,  lakes,  and  dams  within 
±10  NM  of  the  course  line.  These  items  should  be  drawn  to  appropriate  scale  and  match  similar 
features  on  Defense  Mapping  Agency  (DMA)  Tactical  Pilotage  Chart  (TPC)  G-18A. 


REMARKS/LESSONS  LEARNED 

The  arrangement  described  was  satisfactorily  employed  for  Advanced  Technology  Experiment 

#2. 
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SPMR  3.4 


TARGETS 


INTRODUCTION 

The  target  will  be  displayed  only  when  viewed  through  the  simulated  HUD  FOV.  It  will  have 
the  appearance  of  a  SCUD  missile  launcher.  The  target  shall  be  capable  of  being  positioned  at 
multiple  fixed  geographical  coordinates  within  the  simulation  database.  The  missile  launcher 
will  be  visible  when  the  aircraft  arrives  at  the  pull-up  point  for  the  weapon  delivery.  Illumination 
of  the  visual  scene  around  the  target  shall  be  bright  enough  so  that  the  target  is  easily  discernible, 
but  not  so  bright  that  it  distracts  the  pilot. 


REMARKS/LESSONS  LEARNED 

The  arrangement  described  was  satisfactorily  employed  for  Advanced  Technology  Experiment 
#2. 


A-47 


SPMR  3.5 


THREATS 


INTRODUCTION 

The  air-to-air  and  surface-to-air  threat  symbols  shall  only  be  displayed  on  the  HMD  (including 
when  within  ±10  degrees  of  HUD  boresight)  in  both  cockpits.  The  symbols  shall  consist  of  solid 
circles,  solid  equilateral  triangles,  and  solid  squares.  Each  symbol  shall  be  1 4  mr  on  each  side  or 
in  diameter  (as  appropriate)  and  green  in  color.  Brightness  shall  be  adjustable  to  achieve  a  level 
that  permits  the  symbols  to  be  barely  distinguishable  from  the  background  (see  Figure  3.5.1). 

The  threat  symbols  shall  be  positioned  at  fixed  geographic  positions  and  altitudes,  the 
coordinates  will  be  located  in  the  threat  database.  The  type  of  threat  (circle,  triangle,  or  square) 
to  be  displayed  at  designated  points  along  the  route  will  be  identified  with  the  experimental 
trials.  The  threat  symbols  will  be  extinguished  when  the  pilot  activates  the  chaff  dispenser 
paddle  switch  on  the  base  of  the  stick  (see  SPMR  1.1).  If  the  pilot  fails  to  activate  the  chaff 
dispenser  switch,  the  threat  will  stay  illuminated  for  20  seconds. 


Figure  3.5.1  Threat  Shapes 


REMARKS/LESSONS  LEARNED 

Changes  were  made  in  the  way  the  threats  were  presented  to  the  pilots  in  Advanced  Technology 
Demonstration  #  2.  The  intent  of  using  multiple  shapes  to  represent  the  threats  was  to  verify  that 
the  subject  had  a  valid  visual  sighting  (by  requiring  the  subject  to  verbally  identify  the  shape) 
when  activating  the  chaff  switch.  However,  since  the  experimenters  were  able  to  view  a  repeater 
of  the  HMD  video,  they  could  visually  verify  that  the  pilot  was  looking  at  the  target,  making 
verbal  verification  unnecessary.  Additionally,  it  was  discovered  that  for  the  geometric  shapes  to 
be  large  enough  to  identify,  they  had  to  be  so  large  that  they  were  noticed  immediately  whenever 
they  fell  within  the  HMD  FOV.  The  experimenters  felt  that  if  the  threats  were  this  easy  to  see, 
the  task  difficulty  would  not  be  realistic.  Consequently,  the  circle  was  selected  as  the  threat 
symbol.  The  size  and  brightness  of  the  threat  symbol  was  reduced  during  pre-experimental 
sessions  until  the  team  agreed  that  an  appropriate  level  of  discemibility  had  been  achieved. 
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APPENDIX  B 

PILOT  FLYING  EXPERIENCE 
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PILOT  FLYING  EXPERIENCE 


Subject 

# 

Aircraft 

#1 

Hours 

Aircraft 

#2 

Hours 

Aircraft 

#3 

Hours 

Aircraft 

#4 

Hours 

Total 

Hours 

1 

C-141 

80 

F-16 

1350 

1430 

2 

T-38 

1800 

■gl 

1000 

F-15 

250 

3050 

3 

F-16 

700 

OV-IO 

180 

0-37 

500 

0-2 

60 

1440 

4 

F-16 

750 

msm 

820 

F-106 

690 

2260 

5 

B-52 

2500 

T-37 

400 

T-38 

250 

3150 

6 

F-15 

1350 

T-37 

1000 

2350 

|| 

F-16 

220 

A-7D 

550 

770 

T-38 

1200 

F-lll 

900 

2100 

9 

F-16C 

430 

F-4E 

300 

AT-38B 

25 

T-38A 

120 

875 

10 

A- 7 

320 

mm 

700 

T-38 

120 

T-37 

80 

1220 

11 

F-16 

890 

F-106 

600 

A-7 

125 

T-38 

1300 

2915 

12 

F-16 

800 

F-5 

600 

m 

600 

2000 

13 

F-15 

900 

■w 

950 

1850 

14 

F-16 

1200 

OT-37 

500 

OV-IO 

300 

0-2 

300 

2300 

15 

A-7D 

1600 

RF-4C 

870 

T39 

1000 

3470 

16 

TG-7A 

150 

B-52 

2700 

T-41 

150 

3000 

17 

F-lll 

1050 

EF- 111 

1750 

2800 

18 

F-lll 

800 

F-15 

2200 

E-3 

200 

3200 

Total 

Hours 

Total 

Average 

Hours 

Total 

Fighter 

Hours 

Average 

Fighter/ 

Attack 

Hours 

40180 

2618 

26745 

858 

APPENDIX  C 

QUESTIONNAIRES  AND  DATA  COLLECTION  FORMS 
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POST-MISSION  INTERVIEW 


Pilot  Number _  Trial  Number _  Date _  Time _ 

For  trials  1,  2,  7,  and  8  only: 

Use  the  following  numbers  for  recording  pilot’s  responses  to  “rating  scale  questions:” 

“  1  ”  for  “Extremely  Easy” 

“2”  for  “Somewhat  Easy” 

“3”  for  “Neutral” 

“4”  for  “Somewhat  Difficult” 

“5”  for  “Extremely  Difficult” 

1.  Rate  your  acquisition  of  threats  _ 

2.  Rate  your  acquisition  of  the  target  _ 

3.  Rate  the  effort  required  for  weapon  delivery  _ 

4.  Rate  the  effort  required  for  flying  the  airplane  precisely  _ 

Continue  with  questions  below. 

For  ALL  trials: 

5.  Any  comments  on  threat  acquisition? 


6.  Any  comments  on  target  acquisition? 


7  Any  comments  on  weapon  delivery? 


7  Any  comments  on  flying  the  airplane  precisely? 


7  Any  other  comments? 


OUTBRIEF  QUESTIONNAIRES 


Name 


Date 


GENERAL  INSTRUCTIONS:  There  are  two  questionnaires  in  this  packet.  In  the  first 
questionnaire,  you  will  be  asked  to  provide  your  opinion  of  various  aspects  of  the  two  cockpits 
you  have  just  flown.  The  second  questionnaire  asks  you  to  evaluate  the  way  the  study  was 
conducted.  Please  feel  free  to  add  comments  on  any  of  the  questionnaire  topics.  If  you  don’t 
understand  a  question,  we’ll  be  happy  to  explain  it  and/or  discuss  it  with  you.  Thanks  for  your 
participation. 
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HMD  AND  TSD  SYMBOLOGY 


In  this  questionnaire,  please  tell  us  what  you  think  about  the  value  and  acceptability  of  each  individual  element  of 
symbology  used  in  the  HMD  and  TSD.  Fill  in  each  block  below  with  a  number  (including  “+”  or  sign)  or  a 
letter  as  directed.  For  example,  if  you  thought  the  Threat  Cueing  symbol  “substantially  enhanced  your  ability  to 
control  the  aircraft,  enter  a  +2  in  the  “Threat  Cueing  Symbol”  row  in  the  “Control  Aircraft  column.  If  you 
thought  the  Horizon  Line  was  acceptable,  but  would  like  to  suggest  some  changes,  enter  a  “B”  in  the  “Design  & 
Mechanization”  column  AND  write  your  suggestions/comments  on  the  back  of  this  page. 
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Use  this  space  to  provide  comments  about  your  responses  on  the  front  of  this  page. 


COMMENTS  ON  HMD  AND  TSD  SYMBOLOGY 

HMD 

Threat  Cueing  Symbol 
Target/ Waypoint  Cues 
Horizon  Line 

AOA  Indicator 

Master  Arm  Indication 

Mach  Indication 

Airspeed  Indication 

“Orange  Peel”  (Attitude  Indicator) 

Radar  Altitude  Scale 
Barometric  Altitude  Indication 

TSD 

Aircraft  Symbol 
SAM  Threat  Symbol 
Air-to-Air  Threat  Symbol 
Waypoint  Symbol 
Course  Line 
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COCKPIT  EFFECTIVENESS  FOR  SELECTED  MISSION  FUNCTIONS 

Please  tell  us  how  effective  the  baseline  and  IMPACT  cockpits  were  in  enabling  you  to  accomplish 
each  of  the  mission  functions  listed  below  by  placing  the  appropriate  number  (and  sign)  in  each  box. 
For  example,  if  you  thought  the  baseline  was  “Very  Effective”  in  enabling  you  to  determine  threat 
azimuth,  enter  a  +3  in  the  “BASELINE”  row  under  the  “Determine  Threat  Azimuth”  column. 

Please  enter  a  number  in  each  box.  Use  the  back  of  the  page  for  comments. 


IMPACT  TECHNOLOGIES’  CONTRIBUTIONS  TO  MISSION  FUNCTIONS 

Now,  tell  us  what  the  contributions  of  the  HMD,  TSD,  and  directional  audio  were  in  enabling 
you  to  accomplish  the  same  mission  functions.  Again,  enter  the  appropriate  number  (and  sign) 
in  each  box.  (Note  that  the  rating  scale  has  changed.)  For  example,  if  you  thought  the  HMD 
made  a  “Strong  Negative  Contribution”  (e.g.,  really  hampered)  to  your  ability  to  control  the 
aircraft,  enter  a  “-3”  alongside  HMD,  under  the  “Control  Aircraft”  column.  Please  enter  a 
number  in  each  box.  Use  the  back  of  the  page  for  comments. 
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Use  this  space  to  provide  comments  about  your  responses  on  the  front  of  this  page. 

COMMENTS  ON  COCKPIT  EFFECTIVENESS  FOR  SELECTED  MISSION 

FUNCTIONS 

Please  indicate  the  specific  function  you’re  commenting  on  (e.g.,  “Determine  Threat  Elevation”). 


BASELINE 


IMPACT 


COMMENTS  ON  IMPACT  TECHNOLOGIES’  CONTRIBUTIONS  TO  MISSION 

FUNCTIONS 

Please  indicate  the  specific  function  you’re  commenting  on  (e.g.,  “Determine  Threat  Elevation”). 


HMD 


TSD 


DIRECTIONAL  AUDIO 
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ACCEPTABILITY  OF  HMD,  TSD,  AND  DIRECTIONAL  AUDIO 
DESIGN  CHARACTERISTICS 

Please  rate  the  acceptability  of  the  characteristics  of  the  IMPACT  technologies  listed  below.  For 
each  characteristic,  enter  a  check  mark  in  the  box  that  corresponds  to  the  appropriate  rating  category. 
(Check  only  one  box  per  row).  Use  the  back  of  this  page  for  comments. 
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Use  this  space  to  provide  comments  about  your  responses  on  the  front  of  this  page. 

ACCEPTABILITY  OF  HMD,  TSD,  AND  DIRECTIONAL  AUDIO  DESIGN 

CHARACTERISTICS 

HMD 

Comfort 

Field-of-View 
Brightness  and  Contrast 
Focus 
Color 

Alignment  with  “World” 

Dynamics/Update  Rate 

TSD 

Size 

Color  Coding 
Brightness  and  Contrast 
Scaling 
Location 

DIRECTIONAL  AUDIO 

Volume 

Frequency 
Method  (Tone) 
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OTHER  COMMENTS 


Here’s  your  last  chance  to  comment  on  any  aspect  of  either  cockpit  design. 

1 .  What  is  your  overall  impression  of  the  capability  of  the  baseline  cockpit  with  regard  to  threat 
acquisition  and  night  pop-up  attacks? 


2.  What  is  your  overall  impression  of  the  capability  of  the  IMPACT  cockpit  with  regard  to  threat 
acquisition  and  night  pop-up  attacks? 


3.  Are  there  any  other  technology  applications  that  you  think  would  be  useful  in  helping  fighter 
pilots  locate  threats  and  apply  more  aggressive  bombing  tactics  at  night? 
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METHODOLOGY  AND  PROCEDURES  QUESTIONNAIRE 


The  following  questionnaire,  to  be  completed  anonymously  unless  you  care  to  attach  your  name 
for  follow-up  discussion,  was  developed  so  that  you  could  provide  feedback  to  the  project  team 
for  improving  future  studies.  Your  comments  are  valuable  to  us  and  are  greatly  appreciated. 

1 .  Overall,  how  would  you  rate  the  simulator  formats  used  in  this  study? 

BASELINE 

Exceptional  _ 

Good  _ 

Fair  _ 

Poor  _ 

Unacceptable  _ 

Do  you  have  any  other  comments  on  this  topic? 


IMPACT 

Exceptional 

Good 

Fair 

Poor 

Unacceptable 


2.  How  would  you  rate  the  questionnaires  and  interviews  used  in  this  study? 

Exceptional  _ 

Good  _ 

Fair  _ 

Poor  _ 

Unacceptable  _ 

Do  you  have  any  other  comments  on  this  topic? 


3.  Overall,  how  would  you  rate  the  data  collection  procedures? 

Exceptional  _ 

Good  _ 

Fair  _ 

Poor  _ 

Unacceptable  _ 

Do  you  have  any  other  comments  on  this  topic? 


C-Il 


4.  How  would  you  rate  the  quality  of  training  (briefings,  hands-on)? 

Exceptional  _ 

Good  _ 

Fair  _ 

Poor  _ 

Unacceptable  _ 


Do  you  have  any  other  comments  on  this  topic? 


5.  How  would  you  rate  the  “logistics”  (e.g.,  scheduling)  during  your  participation  in  this  study? 


Exceptional 

Good 

Fair 

Poor 

Unacceptable 


Do  you  have  any  other  comments  on  this  topic? 


6.  How  would  you  rate  the  staff  that  conducted  this  study  in  terms  of  knowledge,  preparation, 
and  professionalism? 


Exceptional 

Good 

Fair 

Poor 

Unacceptable 


Do  you  have  any  other  comments  on  this  topic? 


7.  What  parts  of  the  study  do  you  think  should  be  improved  and  why? 


8.  Do  you  have  any  other  comments  about  this  study? 
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Biographical  Data 


Name: _  Date: _ 

Grade:  0-1  0-2  0-3  0-4  0-5  0-6 

Status:  _  (Active  Duty,  Guard,  Reserve,  Retired) 

Organization:  _ 

Duty  Station:  _  Duty  Phone:  _ 

Supervisor’s  name  and  address: _ 

(to  be  used  for  sending  a  letter  of  appreciation  for  participation  in  the  study) 

Aeronautical  Rating:  (please  check  one) 

_ Pilot 

_  Instructor  Pilot 

_  Evaluator  Pilot 

May  we  contact  you  if  additional  information  is  needed?  _ 

Please  list  aircraft  flown  and  approximate  number  of  hours  in  each,  beginning  with  the  most  recent: 
Aircraft:  _  Hours:  _ 

Other  qualifications  (FWFC  graduate,  Test  Pilot  School  graduate,  RTU  Instructor):  _ 

Desert  Storm  experience?  _ Yes  _  No 

Specific  operational  experience:  please  check  experience  level  for  each  category: 

limited  moderate 

Low-level  fighter,  manual  TFR  _  _ 

RWR  and  defensive  threat  reactions  _  _ 

Air-to-ground  weapon  delivery  _  _ 

Advanced  Technologies:  please  list  your  knowledge  of  the  following  technologies: 

no  knowledge  little  knowledge 

Directional  Audio  _  _ 

Helmet-Mounted  Displays/Sights  _  _ 

Large  format  TSDs  _  _ 


extensive 


much  knowledge 
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Real-Time  Data  Probes 


Pilot  Number _  Trial  Number _  Date _  Time 


Medium  altitude  segment 

Identification  of  Threat  1  (circle  one)  sighting  no  sighting 


After  dispensing  chaff  for  Threat  1  SWAT  _  _  _ 

Turning  descent 

Identification  of  Threat  1  (circle  one)  sighting  no  sighting 

After  dispensing  chaff  for  Threat  2  SWAT  _  _  _ 

Low-level  ingress  and  turn  to  IP 

Identification  of  Threat  1  (circle  one)  sighting  no  sighting 

After  dispensing  chaff  for  Threat  3  SWAT  _  _  _ 

Identification  of  Threat  1  (circle  one)  sighting  no  sighting 

After  dispensing  chaff  for  Threat  4  SWAT  _  _  _ 

Weapon  delivery 

After  attacking  target  SWAT  _  _  _ 

Comments:  _ 
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APPENDIX  D 

EXPERIMENTAL  DESIGN 
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EXPERIMENTAL  DESIGN 


Cockpit 


Target 


AA  Threats 


SA  Threats 


Cockpit 


Target 


A  A  Threats 
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Cockpit 


Target 
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Hard,  Easy 

Hard,  Easy 

Easy,  Hard 

Easy,  Hard 

Hard,  Easy 

Easy,  Hard 

Easy,  Hard 

Easy,  Hard 

Easy,  Hard 

Hard,  Easy 

Hard,  Easy 

Hard,  Easy 

Hard,  Easy 

Hard,  Easy 

Easy,  Hard 
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Hard 
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Easy,  Hard 

Easy,  Hard 
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Easy,  Hard 

Hard,  Easy 
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Hard,  Easy 

Easy,  Hard 
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Target 
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Cockpit 


Target 
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SA  Threats 


Cockpit 


Target 
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Hard 

Hard 

Easy 

Easy 

Hard  j 

Hard 
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Hard,  Easy 

Hard,  Easy 

Easy,  Hard 

Hard,  Easy 

Hard,  Easy 

Target 

Easy 

Easy 

Hard 

Hard 

Easy 

Easy 

Hard 

Hard 
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Easy,  Hard 

Hard,  Easy 

Hard,  Easy 

Easy,  Hard 

Easy,  Hard 

Easy,  Hard 

Hard,  Easy 

Hard,  Easy 

SA  Threats 

Hard,  Easy 

Easy,  Hard 
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TARGET  AND  THREAT  LOCATIONS 


|  AIR-TO-AIR  THREAT  LOCATIONS  | 

Threat 

Longitude 

Latitude 

Altitude 

E-1 

-120.992 

36.150 

27500 

E-2 

-121.165 

36.845 

7800 

E-3 

-120.913 

36.203 

17700 

E-4 

-120.763 

36.142 

20500 

E-5 

-120.983 

36.100 

2000 

E-6 

1  -121.198 

36.808 

12500 

E-7 

-120.783 

36.187 

23300 

E-8 

-120.767 

36.153 

13500 

E-9 

-120.935 

36.073 

10000 

E -10 

-121.055 

36.825 

15200 

E-1 1 

-120.800 

36.198 

5000 

E-12 

-120.940 

36.192 

15500 

E-1 3 

-121.152 

36.498 

17500 

E-14 

-121.150 

36.483 

5500 

E-1 5 

-121.047 

36.582 

17500 

E-16 

-120.955 

36.533 

19500 

E-1 7 

-121.138 

36.537 

17500 

E-1 8 

-121.150 

36.477 

10500 

H-1 

-121.137 

36.450 

26600 

H-10 

-120.948 

36.517 

30000 

H-11 

-121.217 

36.720 

25000 

H-1 2 

-121.050 

35.983 

40000 

H-1 3 

-120.758 

36.125 

30000 

H-14 

-120.922 

36.062 

24900  | 

H-1 5 

-120.763 

36.125 

30000 

H-16 

-120.755 

36.138 

28500 

H-1 7 

-120.832 

36.053 

3000  | 

H-1 8 

-120.775 

36.097 

23500 

H-2 

-121.058 

36.413 

30200 

H-3 

-121.203 

36.700 

28900 

H-4 

-121.137 

36.070 

40000 

H-5 

-121.120 

36.460 

29600 

H-6 

-120.967 

36.453 

29600 

H-7 

-121.070 

36.670 

23300 

H-8 

-121.135 

36.052  | 

10000 

H-9 

-120.953 

36.528 

29000 

f  TARGET  LOCATIONS  | 

Target 

Longitude 

Latitude 

Altitude 

T1 

-120.805 

37.177 

1200 

T2 

-120.800 

37.173 

1200 

T3 

-120.810 

37.178 

1200  I 

T4 

-120.797 

37.172 

1200 

T5 

-120.813 

37.180 

1200 

T6 

-120.805 

37.177 

1200 

T7 

-120.768 

37.140 

1200 

T8 

-120.855 

37.180 

1200 

T9 

-120.792 

37.168 

1200 

T10 

-120.818 

37.180 

1200 

Til 

-120.780 

37.158 

1200 

T12 

-120.835 

37.183 

1200 

|  SURFACE-TO-AIR  THREAT  LOCATIONS  j 

Threat 

Longitude 

Latitude 

Altitude 

B-1 

-121.150 

36.950 

1500 

B-10 

-121.137 

36.070 

1500 

B-2 

-121.033 

35.950 

2000 

B-3 

-120.900 

36.550 

3700 

B-4 

-121.017 

36.167 

300 

B-5 

-120.900 

36.833 

2000 

B-6 

-120.767 

36.000 

2000 

B-7 

-121.133 

36.383 

1000 

B-8 

-121.000 

36.200 

1000 

B-9 

-121.078 

36.148 

500 

F-1 

-120.750 

36.150 

1500 

F-10 

-121.130 

36.652 

1500 

F-1 1 

-121.025 

36.592 

4000 

F-1 2 

-121.183 

36.565 

1500 

F-2 

-121.150 

36.600 

2000 

F-3 

-120.967 

36.233 

1000 

F-5 

-120.917 

36.017 

300 

F-6 

-120.983 

36.883 

1000 

F-7 

-120.867 

36.100 

300 

F-8 

-120.967 

36.483 

1000 

F-9 

-121.067 

36.628 

3600 
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APPENDIX  E 

PILOT  COMMENTS 


E-l 


Pilots’  Verbal  Comments  Regarding  the  IMPACT  Configuration 


Pilot 

Rating 

Comments 

1 

0 

Since  we  used  the  same  heads-down  TSD  and  HUD  in  both  cockpits,  I  gave  the  same 
rating  to  "Locate  Target"and  "Deliver  Weapon." 

+ 

The  TSD,  Directional  Audio  and  HMD  combination  made  threat  acquisition  much 
easier.  Also,  the  target  locator  cue  made  target  acquisition  easier. 

+ 

Directional  audio  helps. 

+ 

HMD  arrow  helps. 

0 

Master  moding  should  be  on  the  UFC. 

2 

+ 

A  vast  improvement.  Further  effort  is  needed  so  FLIR  image  does  not  overwhelm 
the  HMD.  I  felt  there  was  too  much  difference  in  brightness  (FLIR  was  too  bright). 

+ 

I  never  would  have  found  the  3rd  threat  w/o  HMD  cueing. 

+ 

Using  the  audio  inputs  takes  some  training.  The  HMD  allows  you  to  reduce  turning 
the  aircraft  as  much  and  to  stay  lower  while  finding  the  threats. 

+ 

Extremely  easy  in  IMPACT. 

+ 

Acquiring  threats  was  more  difficult  w/o  cueing  and  less  time  for  flying. 

+ 

Acquiring  threats  was  extremely  easy  -  Audio  is  not  a  strong  signal  -  Left  or  right 
only  and  still  glance  at  TSD  to  see  threat. 

+ 

HMD  really  helps  stay  on  course  because  it  is  easier  to  see  the  threats. 

- 

It  was  difficult  to  use  the  helmet  and  HUD. 

4- 

Easier  when  using  the  HMD  symbols  (after  6th  trial). 

Pilots’  Verbal  Comments  Regarding  the  IMPACT  Configuration  (cont.) 


Pilot 

Rating 

Comments 

3 

+ 

Threat  acquisition  was  outstanding!  Like  the  “F-15E”  [Baseline]  this  was  no  great 
leap  from  the  Block  40  F-16. 

0 

Not  fully  utilizing  the  HMD  yet. 

+ 

Looked  for  target  in  the  HMD  and  it  worked! 

0 

Tried  to  use  the  HMD,  but  went  back  to  the  HUD. 

+ 

TSD  is  good. 

4 

+ 

This  is  an  improvement  -  a  large  improvement.  Most  of  the  improvement  is  due  to 
the  HMD. 

+ 

I  like  the  cueing,  can  see  the  threats  without  moving  the  aircraft. 

0 

The  target  is  hidden  by  the  target  designator  symbology  sometimes. 

+ 

Used  the  HMD  and  it  helped. 

Tried  to  use  the  HMD  arrow  to  find  the  target,  but  became  confused. 

5 

+ 

Very  effective. 

+ 

Excellent!  Very  effective  for  threat  recognition.  Less  effective  for  SA  when 
transitioning  to  the  HUD  from  the  HMD. 

0 

HMD  is  great  for  threats.  Needs  more  work  in  target  area. 

+ 

HMD  is  easy. 

+ 

Learning  to  turn  head,  using  HMD  symbology. 

- 

The  HMD  diamond  should  be  a  different  color. 

- 

HMD  to  HUD  requires  reacquisition. 

+ 

In  the  IMPACT  cockpit  you  look  outside  for  the  target.  In  the  “F-15E”  [Baseline], 
you  have  to  look  in  to  get  SA. 

0 

Engine  instrument  should  be  on  the  same  side  of  both  cockpits. 

- 

IMPACT  information  sometimes  overloads.  Dimmer  would  help. 

- 

HMD  airspeed  is  not  crisp. 
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Pilots’  Verbal  Comments  Regarding  the  IMPACT  Configuration  (cont.) 


Pilot 

Rating 

Comments 

6 

0 

IMPACT  makes  weapon  delivery  easier,  but  I  still  feel  that  night  low-level  and  night 
pop-up  deliveries  are  a  two  person  kind  of  job. 

+ 

Without  helmet  cueing,  if  you  have  to  look  past  the  beam,  light  pollution  makes  it 
hard  to  see  threats.  Breaking  at  300  feet  on  instruments  is  tough. 

+ 

Pretty  much  a  no-brainer.  Pretty  simple  really  with  Threat  Box  -  line  aids  in 
acquisition. 

0 

On  one  of  the  threats  close  to  six  o’clock,  when  you  move  your  head  quickly,  you 
may  contuse  threat  location  line  with  horizon  line. 

For  the  first  time  today,  I  actually  saw  TD  in  helmet  before  it  was  uncaged  in  the 

HUD;  I  will  start  using  it. 

+ 

Used  helmet  to  see  where  to  roll-out.  Had  more  time  than  would  have  had  otherwise. 

-b 

Difficult  with  big  offset  in  the  “F-15E”  [Baseline] ...  never  know  how  far  off.  The 

i 

HMD  would  have  helped. 

T" 

Used  helmet  target  designator  until  I  got  target  underneath  nose  -  made  it  easier. 

I  picked  it  up  in  the  HMD,  pulled  to  it,  then  lost  it  in  transition  from  HMD  to  HUD. 

HMD  symbology  (alphanumerics)  is  blurry.  _ 


TA-Excellent,  pop-up  better  than  “F-15E”  [Baseline].  _ 


The  HMD  helped  a  lot  in  determining  azimuth  and  elevation  of  the  threat,  which  was 
much  superior  to  the  basic  “F-15E”  [Baseline]. 

The  HMD  in  the  IMPACT  cockpit  provided  superior  threat  information  (e.g.,  SA, 
Azimuth,  and  elevation),  which  was  superior  to  the  “F-15E”  [Baseline]  threat 
information.  During  NAV  and  weapon  delivery,  they  were  equal. 

During  threat  number  one,  I  hit  the  ground  and  was  totally  disoriented. 

Like  IMPACT  better. 

Like  threat  line  coming  at  you  instead  of  somebody  else. 

HMD  is  magical  in  finding  threats. 

Don't  like  chasing  digital  airspeed  indicator. 

Don’t  like  engine  instruments  on  the  left. 


Pilots’  Verbal  Comments  Regarding  the  IMPACT  Configuration  (cont.) 


Pilot 

Rating 

Comments 

9 

+ 

I  like  the  TSD. 

+ 

The  HMD  helps  immensely  in  finding  threats  in  the  IMPACT  cockpit. 

10 

+ 

Much  improved  over  the  other  and  it  allows  one  to  control  the  aircraft  better. 

- 

Threat  #2  had  to  adjust  helmet  (because  of  slippage)  and  never  found  threat. 

+ 

Like  threat  nomenclature  on  HMD. 

11 

+ 

2-D  audio,  HMD,  very  synergistic  for  threat  azimuth.  HMD  with  bore  cross,  target 
locator  line  was  invaluable  for  off-boresight  target  acquisition. 

+ 

Much  better,  felt  almost  as  comfortable  at  night  as  during  day.  Threat  detect  was 
much  more  of  a  non-event. 

- 

Target  locator  line  confused  w/horizon  -  a  dashed  or  double-line  might  help.  I 
initially  turned  based  on  aural  cue  -  peripherally  aware  of  sound. 

0 

Initial  turn  could  have  been  due  to  audio  -  didn't  look  at  TSD. 

+ 

Symbology  was  critical  with  finding  threats  quickly.  Liked  synergism  of  directional 
audio  and  HMD. 

+ 

You  got  to  look  inside  with  the  “F-l 5E”  [Baseline]  for  threats.  With  HMD  you  don't 

+ 

0 

care. 

Directional  audio  just  paid  for  itself.  With  a  combination  of  HMD  and  audio,  you 
could  afford  to  be  a  little  less  precise. 

Target  locator  line  in  HUD  would  help. 

+ 

This  was  the  first  time  I  noticed  the  target  in  the  HMD.  Target  locator  line  is  there. 
Pulled  down  to  it. 

+ 

Geometry  of  pop  (in  “F-15E”  [Baseline])  was  worse  than  I  expected.  Missed  target 
locator  line,  which  was  what  enabled  me  to  find  the  target  quicker  in  the  IMPACT 

+ 

cockpit. 

+ 

The  more  off-boresight  the  target  is,  the  more  the  advantages  show. 

0 

Really  miss  target  locator  line  (in  “F-15E”  [Baseline]). 

Would  rather  have  precision  in  HMD,  if  given  the  choice.  There  needs  to  be  a 

+ 

magnitude  of  the  off  straight  ahead  look.  Maybe  make  target  water  line  proportional. 

The  HMD  makes  finding  threats  easier  -  you  don’t  realize  how  much  until  you  switch 
back  to  the  “F-15E”  [Baseline]. 

Pilots’  Verbal  Comments  Regarding  the  IMPACT  Configuration  (cont.) 


Pilot 

Rating 

Comments 

12 

+ 

More  suitable.  Still  need  audio  warnings.  Much  better  at  maintaining  SA  and 
capability  at  night  in  the  weather. 

- 

The  threat  radar  and  the  missile  flyout  lines  are  confusing.  They  should  be  distinctly 
different. 

+ 

Threat  line  on  HMD  is  great. 

- 

The  sudden  appearance  of  the  horizon  line  and  the  threat  line  together  are  confusing  - 
I  came  back  to  the  TSD.  The  threat  line  should  be  dotted. 

HMD  pointer  needs  to  point  azimuth  first,  then  elevation  second  as  the  pilot  flies. 

+ 

Never  acquired  the  target.  I  like  the  target  arrow  in  HMD  to  help  find  target. 

+ 

The  HMD  target  line  is  very  helpful. 

+ 

Let’s  just  use  the  IMPACT  cockpit  for  the  rest  of  the  time. 

Directional  audio  test  would  be  better  if  there  was  more  background. 

13 

+ 

The  TSD  is  very  good,  no  notable  +/-  with  “F-15E”  [Baseline].  The  Weapon/Threat 
designator  on  HMD  is  excellent! 

+ 

The  threat  acquisition  is  fantastic. 

+ 

I  like  the  HMD  threat  arrow  to  locate  threats. 

- 

Threat  cue  for  #2  was  disorienting  because  of  confusion  with  horizon  line. 

_ 

There  was  confusion  between  the  threat  line  and  the  horizon  line. 

14 

+ 

Great  benefit  over  “F-15E”  [Baseline]  for  determining  threat  azimuth  and  elevation. 

+ 

Good  improvement  over  “F-15E”  [Baseline]. 

- 

Do  not  like  the  engine  instruments  in  IMPACT  -  location  or  format. 
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Pilots’  Verbal  Comments  Regarding  the  IMPACT  Configuration  (cont.) 


Pilot 

Rating 

Comments 

15 

+ 

+ 

Impact  is  four  times  better  at  threat  acquisition.  Impact  is  about  the  same  tor  pop-up 
attacks.  Directional  audio  is  a  great  enhancer.  In  the  Impact  cockpit,  it  is  four  times 
easier  to  detect  threats  and  to  fly  at  low  altitude.  Declutter  the  HUD  during  target 
acquisition. 

Helmet  really  decreases  workload  for  threats. 

4 

4* 

HMD  arrow  really  helps  find  the  threats. 

Audio  is  a  big  help  for  initial  head  movements. 

0 

Too  much  HUD  symbology. 

HUD  declutter  mode  to  erase  some  HUD  symbology  would  be  nice. 

+ 

IMPACT  TSD  is  a  lot  better  than  “F-15E”  [Baseline]  TSD  because  it  is  larger  and 
easier  to  read. 

16 

+ 

Arrows  in  HMD  made  threat  acquisition  significantly  easier. 

+ 

HMD  symbology  helps. 

17 

- 

Threat  #1 :  Helmet  slipped  when  turning.  Threat  #3:  Helmet  slipped  again  -  Hit 
ground  trying  to  put  threat  symbol  on  gun  cross. 

4- 

Would  not  have  found  Threats  #1  and  #2  without  HMD. 

4 

For  Threat  #1,1  didn't  even  use  TSD  as  an  aid. 

4- 

It  is  difficult  to  find  threats  without  HMD  symbology. 

0 

I  look  at  the  TSD  first,  but  I’m  becoming  more  familiar  with  using  the  HMD  to  find 
threats. 

4-  ! 

Learning  curve  is  going  up.  Like  HMD  symbology. 

4 

Never  would  have  found  some  of  the  threats  without  HMD  symbology. 

4 

HMD  guides  a  lot  quicker. 

18 

Threat  acquisition  is  much  improved.  HMD  weapon  delivery  - 1  wished  many  times 
that  I  could  have  turned  the  HMD  off.  For  simple  deliveries,  it  confuses  more  than  it 
helps.  The  transition  between  HMD  and  HUD  is  poor. 

4 

HMD  target  designator  helped  me  find  the  target. 

- 

I  rolled  right  when  target  was  left,  then  I  lost  the  target  in  HMD  and  HUD 
symbology. 
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Pilots’  Verbal  Comments  Regarding  the  “F-15E”  [Baseline]  Configuration 


Pilot 

Rating 

Comments 

1 

0 

Since  we  used  the  same  heads-down  TSD  and  HUD  in  both  cockpits,  I  gave  the  same 
rating  to  "Locate  Target”  and  "Deliver  Weapon." 

0 

It  is  adequate,  but  work  is  required  to  determine  threat  azimuth  and  range.  As  far  as 
night  pop-up  attacks  go,  I  think  operational  pilots  fly  directly  to  the  target  and  do  a 
direct  pop  (leaving  offsets  for  daytime). 

0 

Master-moding  should  be  on  the  UFC. 

0 

Have  to  look  at  the  TEWS  to  find  the  threat. 

2 


+ 


0 


TD  diamond  is  useful. 

Poor  at  best. 

I  do  not  want  to  return  to  the  “F-15E”  [Baseline]  cockpit! 
Avoiding  terrain  was  most  difficult  when  acquiring  threats. 


4 


0 


Disoriented  after  2nd  threat,  used  ADI  to  recover  but  was  confused  about  the  sky  - 

ground  contrast. _ _ _ _ 

The  TEWS  was  quite  useful  for  both  threat  acquisition  and  situation  awareness  (great 

for  SA).  Rest  of  cockpit  was  no  great  leap  from  the  Block  40  F-16. _ 

Threat  Elevation:  In  this  simulation  (i.e.,  no  over-the-shoulder  horizon),  not  knowing 
where  to  look  in  elevation  makes  it  nearly  impossible  to  find  threat  if  the  aircraft  is 
being  maneuvered.  Not  that  bad  in  the  real  world. 


0 


Your  simulation  does  not  do  justice  to  the  “F-15E”  [Baseline]  cockpit.  Although 
IMPACT  technologies  would  help,  the  real  aircraft  is  not  as  bad  as  this  test  made  it 
seem. 


“F-15E”  [Baseline]  engine  instruments  are  easier  to  read  than  those  in  the  IMPACT 
cockpit.  _ _ _ 
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Pilots’  Verbal  Comments  Regarding  the  “F-15E”  [Baseline]  Configuration  (cont.) 


Pilot 

Rating 

Comments 

5 

0 

Adequate  and  effective  for  bombing  when  the  pilot  is  experienced. 

- 

In  the  IMPACT  cockpit,  you  look  outside  for  the  target.  In  the  “F-15E”  [Baseline], 
you  have  to  look  inside  to  get  SA. 

6 

0 

It  is  a  two  person  job.  No  sane  person  would  do  it  single-seat. 

0 

One  of  the  hard  things  is  not  knowing  altitude.  You  can  get  azimuth  off  the  TEWS, 
but  spend  time  figuring  altitude. 

- 

Inbound  in  pop,  you  had  to  think  about  where  the  target  would  be  since  you  had  no 
cues. 

8 

0 

TA  was  improved  compared  to  F-l  1  IE  and  was  poor  compared  to  IMPACT.  Night 
pop  -  OK. 

IMPACT  cockpit  display  and  the  HMD  is  significantly  better  than  the  “F-15E” 
[Baseline]  cockpit,  better  with  HMD. 

“F-15E”  [Baseline]  HUD  needs  algebraic  signs  to  show  dive. 

Controlling  the  A/C  takes  a  significant  amount  of  time  while  trying  to  find  threats. 

9 

+ 

+ 

The  “F-15E”  [Baseline]  TEWS  gives  you  azimuth,  but  does  not  help  with  threat 
elevation. 

Overall  excellent. 

Would  like  a  TSD  in  the  F-l 6. 

10 

0 

It  is  acceptable,  but  needs  to  be  tweaked.  The  HUD  symbology  for  altitude  (above 
and  below  horizon)  needs  to  be  changed. 

I  do  not  like  “F-15E”  [Baseline]  HUD.  There  is  no  perception  of  up  and  down  on 

HUD. 

11 

- 

Only  source  of  azimuth  was  TEWS.  No  source  of  evaluation.  Really  missed  bore 
cross  and  target  locator  like  from  F-l 6. 

- 

Limited  capability  if  you  are  careful  -  accident  waiting  to  happen  if  tapped  -  threat 
detect/react  alone  justifies  HMD. 

- 

Not  having  symbology  makes  threats  much  more  difficult  to  acquire.  Must  look  at  [ 
TEWS,  roll,  look. 

0 

Vectored  aspect  is  where  I  miss  HMD  most. 

12 

- 

Unsuitable  without  audio  warnings. 
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Pilots’  Verbal  Comments  Regarding  the  “F-15E”  [Baseline]  Configuration  (cont.) 


Pilot 

Rating 

Comments 

13 

+ 

The  TSD  is  very  good.  The  HMD  "gray  dot"  is  totally  unacceptable  to  me. 

- 

Overall  cockpit  is  fine.  I  do  not  like  the  HMD  "gray  dot."  I  do  not  like  the  HUD 
Attitude  indicator.  It  does  not  depict  the  difference  between  the  "air"  and  "ground" 
very  well. 

14 

0 

Acceptable. 

15 

0 

Threat  acquisition  cockpit  display  OK,  but  still  hard  to  acquire  threat  at  night.  Night 
pop-up  is  difficult,  but  not  impossible. 

0 

FLIR  glare  made  threat  #1  hard  to  see.  Threats  are  easier  to  see  if  you  don't  have  to 
maneuver  the  A/C. 

- 

In  the  “F-15E”  [Baseline]  it  takes  at  least  1  to  1  1/2  seconds  more  to  find  threats  than 
in  the  IMPACT  cockpit. 

_ 

HUD  declutter  mode  to  erase  some  HUD  symbology  would  be  nice. 

16 

+ 

Great  cockpit-  helped  in  many  ways;  did  not  help  find  target  once  in  pull-up. 

17 

- 

Engine  instruments  should  be  higher. 

18 

■■1 

I  am  not  sure  that  either  setup  is  right,  but  IMPACT  is  moving  in  the  right  direction. 
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Pilots’  Written  Comments  on  the  HMD 


Pilot 

Rating 

Comments 

2 

+ 

Great  aid  in  threat  and  target  acquisition.  I  found  that  I  performed  much  better 
toward  the  end  as  I  learned  how  best  to  use  the  HMD.  I  felt  high  on  the  learning 
curve  for  threat  acquisition,  but  I  had  only  just  begun  to  use  the  HMD  effectively  in 
bombing. 

4 

4 

Determination  of  threat  location  is  the  single  most  valuable  capability  of  the  HMD. 
Another,  which  was  not  tested,  would  be  determination  of  wingman  location.  These 
capabilities  would  be  used  so  often  that  using  the  HMD  would  become  second  nature. 

5 

I  found  that  the  transition  from  the  HMD  to  the  HUD  for  acquiring  the  target  slightly 
degraded  my  ability  to  bomb.  There  was  slight  "confusion"  until  the  target  was 
completely  in  the  HUD. 

9 

+ 

This  by  far  was  the  most  effective  way  to  find  the  threat. 

11 

-f 

Wow  -  however,  the  threat  locator  line  needs  to  be  dashed  to  deconflict  with  horizon 
line.  What  is  the  plan  for  multiple  threats?  I  see  a  potential  mess. 

13 

0 

I  am  not  comfortable  with  this  technology  yet.  I  do  like  the  threat  locator  technology 
with  the  IMPACT  cockpit.  General  Comment:  Overall  I  did  not  use  the  HMD  for 
"anything  forward."  I  only  used  it  to  locate  targets  outside  the  front  field  of  view.  I 
relied  on  the  head  down  or  HUD  information  for  airspeed,  heading,  and  attitude.  I 
think  I  would  need  more  familiarity/training  with  the  HMD  to  gain  more  confidence 
with  its  use. 

14 

As  I  started  to  turn  the  aircraft  to  detect  threats  near  6  o'clock,  I  found  the  threat 
locator  line  somewhat  confusing.  If  I  got  a  majority  of  the  turn  out  of  the  way  and 
used  the  TSD  along  with  the  HMD,  I  did  fine.  However,  just  using  the  HMD  alone 
for  detecting  threats  in  the  rear  hemisphere  was  confusing. 

15 

4 

The  HMD  was  excellent  in  determining  threat  elevation. 

18 

0 

Might  consider  blanking  the  HUD  when  the  FOVs  overlap.  Put  everything  on  the 

HMD  (e.g.,  FLIR  display,  pitch  ladder  when  looking  forward  only). 
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Pilots’  Written  Comments  on  the  TSD 


1  Pilot 

Rating 

Comments 

II 

0 

Might  help  to  make  current  steerpoint  a  recognizable/different  cue.  1  went  backwards 
several  times. 

+ 

Best  SA  device  ever.  Should  be  combined  with  moving  map. 

9 

+ 

The  TSD  was  better  at  telling  you  that  you  are  being  targeted  by  a  threat  than  your 
wingman  would  be.  The  threat  label  was  also  useful. 

11 

4- 

Specifically,  I  liked  the  large  TSD,  especially  compared  to  the  F-16  Block  50D 
horizontal  situation  display  (monochromatic,  small). 

13 

+ 

Is  essential  for  SA.  I  rely  on  the  TSD  when  it  is  available. 

14 

0 

Very  good  for  SA.  However,  the  colors  selected  could  be  better  and  more  targeting 
information  could  be  added.  Additionally,  a  radar  display  and  a  hands  on  throttle  and 
stick  (HOTAS)  capability  to  adjust  the  display  would  be  useful. 

15 

0 

Nice  display,  but  too  much  "heads  down." 
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Pilots’  Written  Comments  on  Directional  Audio 


Pilot 

Rating 

Comments 

5 

+ 

Very  effective. 

6 

+ 

Never  noticed  the  directional  audio  inflight.  Too  many  other  aural  cues  were 
happening  to  notice  it,  except  maybe  subconsciously. 

7 

+ 

True  3-D  audio  would  be  fantastic. 

12 

4“ 

Critical  technology  for  SA  and  effective  control  in  complex  audio  environment. 

15 

4- 

Great  source  of  info  for  initial  target  direction. 

18 

0 

I  never  consciously  used  it,  but  I  do  not  know  if  I  turned  my  head  to  the  right  place 
unconsciously. 

Pilots’  Comments  on  the  HMD  Elements 


Rating  |  Threat  Cueing  Symbol  Comments: _ 

I  think  the  cueing  symbol  (line)  is  too  long.  Move  the  chevrons  closer  to  the  aiming 


It  is  probably  too  small.  The  real  sensor  accuracy  will  not  be  as  good  as  the 
simulation.  Small  cue  leads  to  belief  that  you  will  find  threat  in  that  small  zone. 
Could  lead  to  missing  threat  due  to  focused  search.  Also,  too  bright  for  simulation. 


Sometimes  easy  to  confuse  with  horizon  line  for  6  o'clock  threats.  _ 


Threat  line  needs  to  have  an  urgency-cueing  capability.  (Strobes  for  more  dangerous 
(closer)  threats).  _ _ 


The  HMD  display  could  have  better  resolution  and  fit.  The  general  idea  was  good, 
but  actual  implementation  could  be  better. _ _ _ 


OK.  _ ___ 


Rating  I  Target/Waypoint  Symbol  Comments: _ 


0  Keep  it  common  with  HUD. _ 


0  Make  sure  the  waypoints  go  under  the  plane.  They  should  be  fixed  to  the  ground. 


0  Need  to  be  consistent  -  ordinary  waypoints  0.  See  picture. _ 


0  Show  3-D. 


Rating  Master  Arm  Indication: _ 


Too  small  to  read.  _ _ 


Why  have  it?  It  is  more  clutter. 


Never  saw  it.  _ _ _ 


Could  not  read  it  clearly. 


Airspeed  Indication:  _ _ _ 


Too  small  to  read. 


Stay  common  with  the  HUD.  _ _ _ 


Both  were  slightly  harder  to  read.  _ _ 


Need  an  analog  and  digital  airspeed  indicator.  Also,  need  to  have  a  speed  caret  for 

guidance  on  programmed  speed. _ 

OK,  but  it  would  be  better  to  have  a  tape  display.  _ 


Delete  it.  _ _ _ _ 


Make  it  bigger.  _  _ _ 


Fuzzy  small,  unreadable;  analog  display  would  be  better.  Have  capability  to  bug  a 
desired  speed,  and  show  how  fast  or  slow  you  are  to  the  reference. _ 


Rating  I  Orange  Peel:  •  _ _ _ _ 


0  It’s  not  intuitive.  I  never  used  it. _ 


If  I  tried  to  use  it  while  looking  back,  it  caused  channelized  attention.  _ 


Try  to  get  a  couple  unusual  attitude  situations  to  see  how  the  indicator  operates. 


Lose  the  orange  peel.  _ _ 


Never  used  it.  It  might  be  a  little  small.  _ _ 


Did  not  even  notice  it.  _ _ 


Too  small  to  be  useable. 


Need  different  mechanization  -  it  is  confusing  and  unreadable.  _ 
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Pilots’  Comments  on  the  HMD  Elements  (cont.) 


13 

- 

I  do  not  like  the  "orange  peel"  attitude  system,  but  if  we  must  have  one  I  would  like  it 
to  be  bigger. 

14 

- 

Size  and  location  made  it  difficult  to  use.  I  used  HUD  instead. 

15 

- 

Too  small. 

Pilot 

Rating 

Horizon  Line: 

■ 

Needs  to  have  a  different  texture/look  than  the  threat  cue  line  (I  became  confused 
when  threat  was  near  horizon). 

13 

I  think  it  would  be  better  to  make  the  horizon  line  easily  distinguishable  from  the 
threat  cueing  line.  Perhaps  you  could  also  add  some  slash  marks  in  a  downward 
direction  to  depict  the  ground. 

15 

- 

Change  it:  maybe. 

18 

- 

Delete  it.  It  is  confusing  with  other  solid  lines.  If  you  cannot  delete  it,  make  it 
different. 

Pilot 

Rating 

Aiming  Cross: 

II 

+ 

I  like  it.  It  seems  to  give  a  common  reference  that  keeps  all  the  HMD  symbology  in 
perspective. 

10 

- 

There  is  too  much  trash.  I  prefer  a  simple  aiming  cross. 

Pilot 

Rating 

Mach  Indication: 

4 

0 

Stay  common  with  the  HUD. 

5 

- 

Both  were  slightly  hard  to  read. 

7 

“ 

Do  not  need  it  in  the  HMD. 

11 

- 

Delete  it. 

14 

- 

Could  not  read  clearly. 

15 

- 

Delete  it. 

17 

Fuzzy  small,  unreadable;  analog  display  would  be  better.  Have  capability  to  bug  a 
desired  speed,  and  show  how  fast  or  slow  you  are  to  the  reference. 

Pilot 

Rating 

Radar  Altitude: 

5 

- 

It  was  too  far  from  the  center  of  vision  to  be  used  effectively. 

11 

- 

I  did  not  use  it.  Make  sure  it  is  declutterable. 

15 

" 

Too  small.  Only  one  (radar  or  baro  altitude)  is  required.  Use  baro  above  3000  feet 
AGL. 

Pilot 

Rating 

Baro  Altitude: 

11 

- 

Delete  it. 

15 

- 

Too  small.  Only  one  (radar  or  baro  altitude)  is  required.  Use  baro  above  3000  feet 
AGL. 
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Pilots’  Comments  on  the  TSD  Elements 


Pilot 

Rating 

SAM  Threat  Symbol: 

1 

The  hexagon  symbol  with  the  missile  flying  out  became  slightly  confusing.  (Is  it  a 
SAM  or  an  aircraft?)  Perhaps  we  should  delete  the  missile  inflight  and  leave  the 
hexagon. 

'll 

|B 

If  the  location  data  (e.g.,  range/bearing)  are  reasonably  accurate,  then  it  is  good. 
Otherwise,  it  could  be  (fatally)  misleading. 

7 

“ 

Need  better  indication  of  lock-on  to  discriminate  between  SAMs  and  air-to-air 
threats. 

9 

4* 

Great.  It  adds  SA. 

15 

+ 

It  is  OK. 

Air-to-Air  Threat  Symbol: 

■ 

4- 

If  the  location  data  (e.g.,  range/bearing)  are  reasonably  accurate,  then  it  is  good. 
Otherwise,  it  could  be  (fatally)  misleading. 

5 

0 

It  might  be  helpful  to  have  speed  and  "bad  guy"  symbology  on  the  HMD.  It  wasn't 
until  after  I  acquired  the  "bogie"  did  I  come  back  inside. 

7 

- 

Need  better  indication  of  lock-on  to  discriminate  between  SAMs  and  air-to-air 
threats. 

9 

+ 

Great.  It  adds  SA. 

15 

It  is  OK. 

Bi 

— 1 

Waypoint  Symbol: 

■EH 

a i 

Great.  It  adds  SA.  ; 

15 

0 

Add  information  such  as  time  and  fuel  at  waypoint. 

E231I 

Aircraft  Symbol: 

Great.  It  adds  SA. 

15 

+ 

It  is  OK. 

Pilot 

Rating 

Course  Line: 

9 

+ 

Great.  It  adds  SA. 

15 

4- 

It  is  OK. 
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APPENDIX  F 

STATISTICAL  DATA 
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STATISTICAL  DATA 


1.  Measures  of  Effectiveness 

The  measures  of  effectiveness  that  were  addressed  in  this  experiment  include  threat 
acquisition  time  and  threat  acquisition  success  rate. 

1.1  Threat  Acquisition  Time 

A  2  x  2  x  3  (Cockpit  Configuration  by  Threat  Difficulty  by  Mission  Phase)  repeated 
measures  ANOVA  was  performed  on  the  threat  acquisition  time  data.  A  significant  Cockpit 
Configuration  by  Threat  Difficulty  interaction  was  found,  F(1 , 1 7)  =  8.20,  p  <  .05.  A  simple 
main  effect  analysis  indicated  that  threat  acquisition  time  was  significantly  shorter  for  the 
IMPACT  cockpit  versus  the  baseline  in  the  easy  threat  condition,  F(l,17)  =  56.73,  p  <  .05,  as 
well  as  the  hard  threat  condition,  F(l,17)  =  52.63,  p  <  .05.  The  interaction  was  caused  by  the 
magnitude  of  delta  between  Cockpit  Configuration  as  a  function  of  Threat  Difficulty  (see  Figure 
1). 


Figure  1 .  Mean  Threat  Acquisition  Time 


A  significant  Threat  Difficulty  by  Mission  Phase  interaction  was  also  found  for  threat 
acquisition  time,  F(2,34)  =  15.59,  p  <  .05.  A  simple  main  effect  analysis  indicated  that  threat 
acquisition  time  was  significantly  shorter  for  easy  threats  versus  hard  threats  for  Medium 
Altitude  Cruise,  F(  1 , 1 7)  =  17.20,  p  <  .05,  TFR  Descent,  F(l,17)  =  14.41,  p  <  .05,  and  TFR 
Ingress,  F(l,17)  =  81.47,  p  <  .05.  The  interaction  was  caused  by  the  magnitude  of  delta  between 
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Threat  Difficulty  as  a  function  of  the  Mission  Phase.  No  other  interactions  were  discovered  for 
threat  acquisition  time. 

A  significant  main  effect  for  Cockpit  Configuration  v/as  found  as  a  function  of  threat 
acquisition  time,  F(l,17)  =  80.65,  p  <  .05.  Threat  acquisition  time  was  significantly  shorter  with 
the  IMPACT  cockpit  (M  RMS  =  8.43  sec.)  versus  the  baseline  (M  RMS  =  12.95  sec.). 

A  significant  main  effect  for  Threat  Difficulty  was  found  as  a  function  of  threat  acquisition 
time,  F(l,17)  =  75.46,  p  <  .05.  Threat  acquisition  was  significantly  shorter  with  the  easy  threats 
(RMS  M  =  8.28  sec.)  versus  hard  threats  (M  RMS  =  13.10  sec.). 

A  significant  main  effect  for  Mission  Phase  was  found  as  a  function  of  threat  acquisition 
time,  F(l,17)  =  9.84,  p  <  .05.  It  appears  that  threat  acquisition  time  was  longer  during  the 
Medium  Altitude  Cruise  Phase  (M  RMS  =  12.15  sec.)  versus  the  TFR  Descent  (M  RMS  =  10.52 
sec.)  or  the  TFR  Ingress  (M  RMS  =  10.04  sec.).  No  other  main  effects  were  discovered.  See 
Table  1  for  a  summary  of  threat  acquisition  time  results,  in  which  the  statistically  significant  p- 
values  are  in  bold  type. 


Table  1.  Threat  Acquisition  Time  Results 


Source 

F 

Significance 

Configuration 

F(l,17)  =  80.65 

p  <  .05 

Threat  Difficulty 

F(l,17)  =  75.46 

p  <  .05 

Phase 

F(2,34)  =  9.84 

p  <  .05 

Configuration  by  Threat  Difficulty 

F(l,17)  =  8.20 

p  <  .05 

Configuration  by  Phase 

F(2,34)=  1.18 

p  >  .05 

Threat  Difficulty  by  Phase 

F(2,34)=  15.59 

p  <  .05 

Configuration  by  Threat  Difficulty  by  Phase 

F(2,34)  =  0.20 

p  >  .05 

1.2  Threat  Acquisition  Success  Rate 

A  2  x  2  x  3  (Cockpit  Configuration  by  Threat  Difficulty  by  Mission  Phase)  repeated 
measures  ANOVA  was  performed  on  the  threat  acquisition  success  rate  data.  A  significant  main 
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effect  was  found  for  Cockpit  Configuration,  F(l,17)  =  32.89,  p  <  .05.  As  shown  in  Figure  2, 
approximately  20%  more  threats  were  acquired  with  the  IMPACT  cockpit  than  with  the  baseline 
throughout  the  entire  mission. 


Figure  2.  Mean  Threat  Acquisition  Success  Rate 


A  significant  Threat  Difficulty  by  Mission  Phase  interaction  was  discovered,  F(2,34)  =  7.34, 
p  <  .05.  A  simple  main  effect  analysis  indicated  that  success  rate  was  significantly  higher  for 
easy  threats  versus  hard  threats  during  TFR  Ingress  only,  F(l,17)  =  15.15,  p  <  .05.  No  other 
interactions  were  discovered  for  threat  acquisition  success  rate. 

A  significant  main  effect  for  Threat  Difficulty  was  found  as  a  function  of  threat  acquisition 
success  rate,  F(  1 , 1 7)  =  6.85,  p  <  .05.  Success  rate  was  significantly  higher  for  easy  threats  (M 
RMS  =  91.60%)  versus  hard  threats  (M  RMS  =  83.91%). 

A  significant  main  effect  for  Mission  Phase  was  found  as  a  function  of  threat  acquisition 
success  rate,  F(l,17)  =  4.86,  p  <  .05.  It  appears  that  threat  acquisition  success  rate  increased 
significantly  from  one  mission  phase  to  the  next.  Mean  RMS  for  Medium  Altitude  Cruise  = 
82.26%,  mean  RMS  for  TFR  Descent  =  86.94%,  and  mean  RMS  for  TFR  Ingress  =  90.90%.  No 
other  main  effects  were  discovered.  For  a  summary  of  threat  acquisition  success  rate  results,  see 
Table  2. 
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Table  2.  Threat  Acquisition  Success  Rate  Results 


Source 

F 

Significance 

Configuration 

F(l,17)  =  32.89 

p  <  .05 

Threat  Difficulty 

F(l,  1 7)  =  6.85 

p  <  .05 

Phase 

F(2,34)  =  4.86 

p  <  .05 

Configuration  by  Threat  Difficulty 

F(l,17)=  1.65 

p  >  .05 

Configuration  by  Phase 

F(2,34)  -  0.47 

p  >  .05 

Threat  Difficulty  by  Phase 

F(2,34)  =  7.34 

p  <  .05 

Configuration  by  Threat  Difficulty  by  Phase 

F(2,34)  =  2.68 

p  >  .05 

2.  Measures  of  Workload 

Workload  was  measured  using  SWAT.  A  2  x  2  x  3  (Cockpit  Configuration  by  Threat 
Difficulty  by  Mission  Phase)  repeated  measures  ANOVA  was  performed  on  the  SWAT  data.  A 
significant  Configuration  by  Threat  Difficulty  interaction  was  discovered,  F(  1 , 1 7)  =  5.48,  p  < 
.05.  A  simple  main  effect  analysis  indicated  that  SWAT  scores  were  significantly  lower  for  the 
IMPACT  cockpit  versus  the  baseline  in  the  easy  threat  condition,  F(1 , 1 7)  =  58.21,  p  <  .05,  as 
well  as  the  hard  threat  condition,  F(  1 ,1 7)  =  12.69,  p  <  .05.  The  interaction  appeared  to  be  caused 
by  the  magnitude  of  the  delta  between  Cockpit  Configurations  as  a  function  of  Threat  Difficulty 
(see  Figure  3). 
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Figure  3.  Mean  SWAT  Scores 
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A  significant  Threat  Difficulty  by  Mission  Phase  interaction  was  also  found  for  SWAT 
scores,  F(2,34)  =  9.14,  p  <  .05.  A  simple  main  effect  analysis  indicated  that  SWAT  scores  were 
significantly  lower  for  easy  threats  versus  hard  threats  during  Medium  Altitude  Cruise,  F(  1 , 1 7)  = 
17.38,  p  <  .05,  and  TFR  Ingress,  F(1 ,1 7)  =  70.23,  p  <  .05,  only.  No  other  interactions  were 
found  for  SWAT  scores. 

A  significant  main  effect  for  Cockpit  Configuration  was  found  as  a  function  of  SWAT 
scores,  F(  1,17)  =  45.18,  p  <  .05.  Scores  were  significantly  lower  for  the  IMPACT  cockpit  (M  = 
33.20)  versus  the  baseline  (M  =  50.78). 

A  significant  main  effect  for  Threat  Difficulty  was  found  as  a  function  of  SWAT  scores, 
F(  1 , 1 7)  =  66.68,  p  <  .05.  SWAT  scores  were  significantly  lower  for  easy  threats  (M  =  34.23) 
versus  hard  threats  (M  =  49.75).  No  other  main  effects  were  found. 

For  the  Weapon  Delivery  Phase,  a  separate  2x2  (Cockpit  Configuration  by  Target 
Difficulty)  repeated  measures  ANOVA  was  performed  on  the  SWAT  data.  No  significant 
differences  were  found  in  SWAT  scores  regarding  Target  Difficulty  or  Cockpit  Configuration. 
See  Table  3  for  a  summary  of  SWAT  results. 


Table  3.  SWAT  Score  Results 


Source 

F 

Significance 

Configuration 

OO 

II 

p  <  .05 

Threat  Difficulty 

F(l,17)  =  66.68 

p  <  .05 

Phase 

F(2,34)  =  2.72 

p>  .05 

Configuration  by  Threat  Difficulty 

F(1 ,17)  =  5.48 

p  <  .05 

Configuration  by  Phase 

F(2,34)  =  0.78 

p>  .05 

Threat  Difficulty  by  Phase 

F(2,34)  =  9.14 

p  <  .05 

Configuration  by  Threat  Difficulty  by  Phase 

F(2,34)  =  0.71 

p  >  .05 

Configuration  (Weapon  Delivery) 

F(  1 , 1 7)  =  0.12 

p>  .05 

Target  Difficulty  (Weapon  Delivery) 

F(  1 , 1 7)  =  0.16 

p  >  .05 

Configuration  by  Target  Difficulty 

F(  1 , 1 7)  =  0.25 

p>  .05 

F-6 


3.  Measures  of  Situation  Awareness 


A  SWORD  form  was  given  to  the  pilots  to  evaluate  situation  awareness  for  each  of  the 
following  Mission  Functions:  threat  acquisition,  target  acquisition,  and  fly/navigate.  The  results 
are  shown  in  Figure  4. 


THREAT  TARGET  FLY/NAVIGATE 

ACQUISITION  ACQUISITION 


IVfcsion  Functions _ 

Figure  4.  Mean  SWORD  Situation  Awareness  Ratings 

A  2  x  3  (Cockpit  Configuration  by  Mission  Function)  repeated  measures  ANOVA  was 
performed  on  SWORD  data.  A  significant  Cockpit  Configuration  by  Mission  Function 
interaction  was  discovered,  F(2,34)  =  29.98,  p  <  .05.  A  simple  main  effect  analysis  indicated 
that  SA  was  rated  relatively  higher  for  the  IMPACT  cockpit  during  threat  acquisition,  F(1 ,17)  = 
76.43,  p  <  .05,  and  target  acquisition,  F(l,17)  =  15.90,  p  <  .05,  only.  There  was  no  statistically 
significant  difference  between  cockpits  for  SA  during  the  fly/navigate  function.  For  a  summary 
of  SWORD  results,  see  Table  4. 


Table  4.  SWORD  Rating  Results 


Source 

F 

Significance 

Configuration 

T1 

5 

ii 

oo 

00 

On 

p  <  .05 

Function 

F(2,34)  =  6.25 

p  <  .05 

Configuration  by  Function 

F(2,34)  =  29.98 

p  <  .05 

4.  Measures  of  Performance 


The  performance  measures  collected  and  analyzed  in  this  experiment  include  flight 
performance  during  threat  acquisition  and  flight  performance  during  weapon  delivery . 

4.1  Flight  Performance  During  Threat  Acquisition 

All  flight  path  tracking  analyses  were  conducted  using  a  2  x  2  (Cockpit  Configuration  by 
Threat  Difficulty)  repeated  measures  ANOVA.  Separate  analyses  were  conducted  based  on 
Mission  Phase  (Medium  Altitude  Cruise,  TFR  Descent,  and  TFR  Ingress).  Separate  analyses 
were  performed  because  each  phase  had  different  MOPs,  as  well  as  different  altitude  and 
airspeed  criteria  specified.  Only  the  data  collected  during  threat  engagements  were  analyzed. 

In  the  Medium  Altitude  Cruise  Phase,  deviations  from  planned  altitude,  course,  and 
groundspeed  were  analyzed.  A  significant  Configuration  by  Threat  Difficulty  interaction  was 
found,  F(1 , 1 7)  =  5.21,  p  <  .05.  A  simple  main  effect  analysis  indicated  that  altitude  deviation 
was  significantly  lower  for  the  IMPACT  cockpit  (M  RMS  =  107.95  ft.)  versus  the  baseline  (M 
RMS  =  285.68  ft.)  in  the  easy  threat  condition  only,  F(l,17)  =  6.42,  p  <  .05.  No  significant 
effects  were  found  between  the  IMPACT  cockpit  and  the  baseline  in  the  hard  threat  condition. 

Also  in  the  Medium  Altitude  Cruise  Phase,  a  significant  main  effect  for  Cockpit 
Configuration  as  a  function  of  course  deviation  was  found,  F(l,17)  =  43.18,  p  <  .05.  Pilots 
maintained  course  significantly  better  with  the  IMPACT  cockpit  versus  the  baseline  regardless  of 
threat  difficulty  (see  Figure  5). 


Cockpit  Configuration 


Figure  5.  Mean  RMS  Course  Deviation  During  Medium  Altitude  Cruise 
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For  groundspeed,  no  significant  differences  were  found  between  the  IMPACT  cockpit  and 
the  baseline  cockpit. 

Significant  main  effects  were  found  for  Threat  Difficulty  as  a  function  of  both  course 
deviation,  F(l,17)  =  5.9,  p  <  .05  and  groundspeed  deviation  F(1 ,1 7)  =  19.7,  p  <  .05.  Smaller 
course  deviations  were  found  in  the  easy  threat  condition  (M  RMS  =  6.85°)  than  the  hard  threat 
condition  (M  RMS  =  11.58°).  Smaller  groundspeed  deviations  were  found  in  the  easy  threat 
condition  (M  RMS  =  18.50  kts.)  than  the  hard  threat  condition  (M  RMS  =  49.99  kts.).  No  other 
interaction  or  main  effects  were  discovered.  A  summary  of  flight  performance  results  for  the 
Medium  Altitude  Cruise  Phase  is  provided  in  Table  5. 


Table  5.  Medium  Altitude  Cruise  Flight  Performance  Results 


Source 

F 

Significance 

Configuration  (Altitude  Deviation) 

F(l,17)=  1.00 

p  >  .05 

Configuration  (Course  Deviation) 

F(1 ,1 7)  =  43.13 

p  <  .05 

Configuration  (Groundspeed  Deviation) 

F(  1 , 1 7)  =  0.05 

p  >  .05 

Threat  Difficulty  (Altitude  Deviation) 

F(1 ,17)  =  9.29 

p  <  .05 

Threat  Difficulty  (Course  Deviation) 

F(l,17)  =  5.89 

p  <  .05 

Threat  Difficulty  (Groundspeed  Deviation) 

F(l,17)  =  19.66 

p  <  .05 

Configuration  by  Threat  Difficulty  (Altitude  Deviation) 

F(1 ,1 7)  =  5.21 

p  <  .05 

Configuration  by  Threat  Difficulty  (Course  Deviation) 

F(l,17)  =  3.70 

p  >  .05 

Configuration  by  Threat  Difficulty  (Groundspeed 
Deviation) 

F(l,17)  =  2.30 

p  >  .05 

Deviation  from  planned  dive  angle  was  analyzed  in  the  TFR  Descent  Phase.  No  interaction 
effects  or  main  effects  were  found. 

In  the  TFR  Ingress  Phase,  deviations  from  planned  course,  groundspeed,  and  terrain 
following  were  analyzed.  Significant  main  effects  for  Cockpit  Configuration  were  found  for 
course  deviation,  F(l,17)  =  10.97,  p  <  .05,  and  TFR  deviation,  F(l,17)  =  8.19,  p  <  .05.  Course 
deviation  was  significantly  smaller  with  the  IMPACT  cockpit  versus  the  baseline.  Also,  TFR 
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deviation  was  significantly  less  with  the  IMPACT  cockpit  versus  baseline.  No  significant 
differences  were  found  between  cockpit  configurations  for  groundspeed  deviation.  See  Table  6 
for  mean  RMS  results. 

Table  6.  Significant  Cockpit  Configuration  RMS  Results  for  the  TFR  Ingress  Phase 


Variable 

Baseline 

IMPACT 

Course  Deviation 

M  =  4.33° 

M  =  3.00° 

TFR  Deviation 

M  =  23.44  mr 

M  =  1 9.04  mr 

Significant  main  effects  were  found  for  Threat  Difficulty  as  a  function  of  course  deviation, 
F(l,17)  =  71.73,  p  <  .05,  groundspeed  deviation,  F(l,17)  =  19.34,  p  <  .05,  and  TFR  deviation, 
F(1 ,1 7)  =  49.33,  p  <  .05.  See  Table  7  for  mean  RMS  results.  No  other  interaction  effects  or 
main  effects  were  discovered.  See  Table  8  for  a  summary  of  flight  performance  results  during 
TFR  Ingress. 

Table  7.  Significant  Threat  Difficulty  RMS  Results  for  the  TFR  Ingress  Phase 


Variable 

Easy  Threats 

Hard  Threats 

Course  Deviation 

M=  1.66° 

M  =  5.74° 

Groundspeed  Deviation 

M=  15.25  kts 

M  =  24.55  kts 

TFR  Deviation 

M=  16.06  Mr 

M  =  26.65  Mr 
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Table  8.  TFR  Ingress  Flight  Performance  Results 


Source 

F 

Significance 

Configuration  (Course  Deviation) 

F(l,17)=  10.97 

p  <  .05 

Configuration  (Groundspeed  Deviation) 

F(l,17)  =  0.01 

p>  .05 

Configuration  (TFR  Deviation) 

F(1 ,17)  =  8.19 

p  <  .05 

Threat  Difficulty  (Course  Deviation) 

-4 

II 

'-4 

Lk> 

p  <  .05 

Threat  Difficulty  (Groundspeed  Deviation) 

F(l,17)=  19.34 

p  <  .05 

Threat  Difficulty  (TFR  Deviation) 

F(l,17)  =  49.33 

p  <  .05 

Configuration  by  Threat  Difficulty  (Course  Deviation) 

F(l,17)  =  0.91 

p  >  .05 

Configuration  by  Threat  Difficulty  (Groundspeed 
Deviation) 

F(1 ,17)  =  0.69 

p>  .05 

Configuration  by  Threat  Difficulty  (TFR  Deviation) 

'O 

o 

li 

p>  .05 

4.2  Flight  Performance  During  Weapon  Delivery 

A  weapon  delivery  analysis  was  conducted  using  a  2  x  2  (Cockpit  Configuration  by  Target 
Difficulty)  repeated  measures  ANOVA.  Deviations  from  planned  vertical  flight,  attack  course, 
and  groundspeed  were  analyzed  in  the  Weapon  Delivery  Phase.  A  significant  main  effect  was 
found  for  Target  Difficulty  for  vertical  flight  deviation  only,  F(l,17)  =  7.97,  p  <  .05.  In  this  case, 
pilots  had  significantly  less  vertical  flight  deviation  with  the  easy  target  condition  (M  RMS  = 
315.42  ft.)  than  with  the  hard  target  condition  (M  RMS  =  503.28  ft.).  No  other  main  effects  or 
interactions  were  found  for  attack  course  deviation  or  groundspeed  deviation.  In  addition,  no 
significant  differences  were  found  between  the  cockpit  configurations  for  any  of  the  flight 
performance  variables.  See  Table  9  for  a  summary  of  flight  performance  results  during  the 
Weapon  Delivery  Phase. 
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Table  9.  Weapon  Delivery  Flight  Performance  Results 


Source 

F 

Significance 

Configuration  (Vertical  Flight  Deviation) 

F(l,17)=  1.74 

p  >  .05 

Configuration  (Course  Deviation) 

F(l,17)  =  0.93 

p>  .05 

Configuration  (Groundspeed  Deviation) 

F(l,17)  =  0.16 

p>  .05 

Target  Difficulty  (Vertical  Flight  Deviation) 

-0 

11 

VO 

p  <  .05 

Target  Difficulty  (Course  Deviation) 

F(l,17)=  1.93 

p  >  .05 

Target  Difficulty  (Groundspeed  Deviation) 

F(l,17)  =  2.69 

p  >  .05 

Configuration  by  Target  Difficulty  (Vertical  Flight 
Deviation) 

F(l,17)  =  0.00 

p  >  .05 

Configuration  by  Target  Difficulty  (Course 
Deviation) 

F(  1,17)  —  0.02 

p  >  .05 

Configuration  by  Target  Difficulty  (Groundspeed 
Deviation) 

F(l,17)  =  0.09 

p  >  .05 

5.  Subjective  Data 

Frequencies  and  mean  ratings  were  calculated  for  all  questionnaire  items  employing  rating 
scales.  Complete  responses  to  open-ended  questions  and  other  pilot  comments  collected  are 
included  in  Appendix  E.  Both  post-mission  and  post-experiment  questionnaires  were  used  to 
address  the  overall  contributions  as  well  as  the  specific  individual  contributions  of  each  advanced 
technology  element. 

5.1  Threat  Acquisition  Effectiveness 

The  pilots  were  asked  for  their  assessment  of  the  two  cockpits  in  threat  acquisition.  Figure 
6  illustrates  the  difficulty  ratings  given  for  threat  acquisition  with  respect  to  cockpit 
configuration.  Threat  acquisition  in  the  IMPACT  cockpit  was  rated  easier  than  in  the  baseline 
cockpit  by  the  pilots. 
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Figure  6.  Post-Mission  Pilot  Difficulty  Ratings  for  Threat  Acquisition 


For  threat  acquisition,  16  out  of  18  pilots  specifically  commented  that  IMPACT  was  a  vast 
improvement  over  the  baseline.  This  was  predominantly  due  to  the  HMD  threat  cueing 
capability.  Conversely,  six  out  of  1 8  pilots  specifically  commented  that  threat  acquisition  in  the 
baseline  was  more  difficult,  mainly  because  of  the  lack  of  threat  elevation  cueing. 

Threat  acquisition  was  broken  into  three  elements:  determination  of  azimuth,  determination 
of  elevation,  and  determination  of  range.  Figure  7  depicts  the  overall  cockpit  effective  for  these 
tasks. 


Figure  7.  Average  Cockpit  Effectiveness  Ratings  for  Determining  Threat  Location 

The  pilots  were  then  asked  to  give  their  assessment  of  the  contribution  of  each  of  the 
technologies  to  the  threat  acquisition  elements.  Figure  8  depicts  their  responses. 
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Figure  8.  Contributions  of  Technologies  to  Threat  Localization 

The  HMD  was  rated  by  the  pilots  as  making  its  strongest  contribution  in  determining 
azimuth  and  elevation.  The  TSD  made  its  largest  contributions  in  threat  azimuth  and  range  and 
the  directional  audio  made  a  moderate  contribution  in  azimuth  only. 

Thirteen  out  of  1 8  pilots  specifically  commented  that  the  HMD  improved  threat  acquisition. 
Figures  9  and  10  illustrate  the  specific  contribution  made  by  the  HMD  threat  cueing  symbology 
in  determining  threat  azimuth  and  elevation,  respectively.  In  addition,  the  ratings  indicate  that 
additional  symbology,  not  necessarily  related  to  threat  acquisition,  did  not  have  a  degrading 
effect.  However,  six  pilots  did  comment  regarding  an  occasional  conflict  between  the  HMD 
horizon  line  and  the  HMD  threat  locator  line.  If  the  threat  was  at  the  same  altitude  as  the  aircraft, 
both  lines  would  overlap  in  the  HMD.  The  only  symbology  that  then  remained  to  indicate  the 
threat  location  was  two  chevrons  displayed  at  the  outer  edge  of  the  HMD  FOV. 


Figure  9.  Contributions  of  HMD  Elements  to  Determining  Threat  Azimuth 
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Figure  10.  Contributions  of  HMD  Elements  to  Determining  Threat  Elevation 

As  mentioned  previously,  the  TSD  was  assessed  favorably  in  the  determination  of  threat 
azimuth  and  range.  The  positive  rating  was  most  likely  due  to  the  pilots’  ability  to  scan  a  large 
scale  presentation  of  the  threat  location.  None  of  the  pilots  had  negative  comments  concerning 
the  TSD,  however  the  following  changes  were  suggested: 

•  Provide  a  differentiable  symbol  separating  the  current  steerpoint  from  past  or  future 
steerpoints 

•  Combine  the  display  with  a  moving  map 

•  Provide  more  targeting  information 

Figures  11,  12,  and  13  illustrate  the  specific  contributions  of  TSD  symbology  in 
determining  threat  azimuth,  elevation,  and  range.  Of  the  three  tasks  required  to  acquire  a  threat, 
determination  of  elevation  using  the  TSD  was  given  the  lowest  ratings  (See  Figure  12). 
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Finally,  directional  audio  was  rated  the  lowest  of  the  three  advanced  technologies  for  threat 
acquisition.  This  was  probably  because  of  the  limited  capability  of  the  audio  system  (azimuth 
only),  as  well  as  the  lack  of  pilot  experience  with  advanced  audio  capabilities.  Of  the  three 
technologies,  directional  audio  seemed  to  be  the  most  foreign  to  the  pilots  as  indicated  by  their 
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wide  range  of  comments.  Five  pilots  indicated  that  the  directional  audio  was  helpful  in 
determining  initial  head  movements.  However,  three  pilots  commented  that  they  were  not 
always  consciously  aware  of  the  audio  cueing  and  would  require  more  training  before  using  it. 

5.2  Weapon  Delivery  Effectiveness 

•  Figure  14  illustrates  the  post-mission  subjective  ratings  given  by  the  subjects  with  respect  to 

configuration  and  target  acquisition  difficulty. 


Figure  14.  Post-Mission  Difficulty  Ratings  for  Target  Acquisition 

Target  acquisition  was  rated  slightly  easier  in  the  IMPACT  cockpit  than  in  the  baseline 
cockpit.  The  following  pilot  comment  provides  a  possible  reason  for  the  rating:  “I  used  (the) 
helmet  to  see  when  to  roll  out . . .  (I)  had  more  time  than  would  have  had  otherwise.” 

Figure  15  depicts  cockpit  effectiveness  ratings  for  the  target  acquisition  and  weapon 
delivery  functions.  Again,  the  IMPACT  cockpit  was  rated  higher  than  the  baseline  cockpit  in 
both  areas. 


* 
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Figure  15.  Average  Cockpit  Effectiveness  Ratings 
for  Target  Acquisition  and  Weapon  Delivery 


As  expected,  the  pilots  felt  the  HMD  and  TSD  were  the  two  technologies  making  the  largest 
contributions  to  the  weapon  delivery  (directional  audio  was  not  used  for  weapon  delivery). 
However,  pilot  comments  were  mixed.  Six  pilots  commented  that  the  HMD  was  useful  in 
acquiring  the  target  earlier  in  the  engagement,  thereby  allowing  more  time  for  planning  the 
attack.  Four  pilots  commented  that  the  transition  from  the  HMD  to  the  HUD  was  difficult  and 
confusing.  Figure  16  shows  that  the  Target/Waypoint  Cue  in  the  HMD  made  a  slightly  positive 
contribution  to  weapon  delivery. 


Figure  16.  Contributions  of  HMD  Elements  to  Weapon  Delivery 

Because  the  weapon  delivery  was  predominantly  a  visual  task,  it  was  understandable  that 
the  TSD  was  rated  slightly  lower  in  its  contribution  (see  Figure  17). 
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Figure  17.  Contributions  of  TSD  Elements  to  Weapon  Delivery 

5.3  Aircraft  Control 

Pilots  were  asked  to  assess  the  cockpits  for  flying  difficulty.  Figure  18  shows  that  they 
found  both  cockpits  moderately  easy  to  fly  with  IMPACT  being  slightly  easier. 


Figure  18.  Post-Mission  Difficulty  Ratings  for  the  Flying  Function 

Figure  19  shows  similar  results  when  the  pilots  were  asked  for  their  cockpit  effectiveness 
ratings  on  the  post-experiment  questionnaire.  IMPACT  was  rated  slightly  more  effective. 


F-19 


Very 

Effective 


Moderately 

Effective 


Slightly 

Effective 


Ineffective 


Baseline  IMPACT 

Cockpit 


Figure  19.  Average  Effectiveness  Ratings  by  Cockpit  for  the  Flying  Function 


Several  elements  of  the  HMD  were  identified  as  providing  small  enhancements  to  flying. 
The  elements  are  shown  in  Figure  20.  Note  that  not  one  element  was  rated  above  “Moderately 
Enhanced.” 


Figure  20.  Average  Ratings  of  HMD  Display  Elements  for  the  Flying  Function 

Several  of  the  elements  of  the  TSD  were  also  identified  as  providing  small  enhancements  to 
flying  and  are  shown  in  Figure  21. 
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Figure  21 .  Average  Ratings  of  TSD  Elements  for  the  Flying  Function 


Note  that  the  only  head  down  navigation  references  in  the  cockpit  (Waypoint  Symbol  and 
Course  Line)  were  rated  as  having  almost  no  effect  on  flying. 


The  rated  contributions  of  the  three  technologies  toward  situation  awareness  are  shown  in 
Figure  22  with  the  TSD  making  the  largest  contribution.  Six  out  of  18  pilots  specifically 
identified  the  IMPACT  TSD  as  providing  the  greatest  level  of  situation  awareness. 


Figure  22.  Contributions  of  Technologies  to  Maintaining  Situation  Awareness 


5.4  Hardware  and  Design  Characteristics 


The  design  characteristics  of  the  three  technologies  were  examined  to  determine  the  pilots’ 
acceptance.  Figure  23  shows  these  acceptability  ratings.  The  TSD  was  the  closest  to  being  rated 
completely  acceptable. 
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Figure  23.  Average  Acceptability  Ratings  of  Design  Characteristics 

The  following  charts  show  the  acceptability  ratings  for  the  design  of  the  individual 
technologies.  As  shown  in  Figure  24,  the  comfort  of  the  helmet  received  low  ratings.  The  low 
ratings  were  not  surprising  because  the  helmet  was  a  prototype  required  to  fit  multiple  subjects 
under  test  conditions.  Furthermore,  all  aspects  of  the  HMD  were  difficult  to  setup,  and  it  was  a 
challenge  for  the  experimenters  to  control  extraneous  effects  in  a  reliable  fashion. 


Figure  24.  Average  Acceptability  Ratings  of  the  HMD  Design  Characteristics 
All  the  TSD  design  characteristics  were  rated  almost  completely  acceptable  (see  Figure  25). 


Figure  25.  Average  Acceptability  Ratings  of  the  TSD  Design  Characteristics 
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Finally,  Figure  26  shows  that  the  pilots  gave  a  “Moderately  Acceptable”  rating  for  the 
directional  audio. 


Figure  26.  Average  Acceptability  Rating  of  the  Directional  Audio  Design  Characteristics 


